My latest Just Chillin' was made for Hannah, and it has the same removable pompom as the original Just Chillin' hat pattern sample made for Mallory.
Within my Just Chillin' pattern there are instructions on how I made a removable pompom on both of these hats...
Each of these (real) fur pompoms* came attached to a key ring with an small elastic loop. If yours has one as well, you could make your fur pompom removable for a different look, and to wash your hat. First, I sewed a large flat button inside each hat.
Then, I used a crochet hook to pull the pompom’s elastic loop (detached from key ring) inside the hat near the button. Back on the inside, I pulled the loop over the button to attach the pompom.
I have since given some thought as to how I would make my own removable yarn pompom. Typically, a yarn pompom can be sewn on a hat, and the hat can be washed with the pompom attached. Even so, I can imagine a hat with interchangeable pompoms, or a case when you might want to wear a hat with or without a (yarn) pompom.
My recommendation is to use a piece 1/8" wide elastic, or maybe a piece of elastic cord, about 8" long. Normally when I make a yarn pompom, I leave the tying yarn quite long and use it to attach the pompom to the hat.
To convert such a pompom to a removable one, thread an 8" length of narrow elastic or heavy elastic cord on a tapestry needle and draw the needle in and out of the pompom below the tying yarn.
Similar to the removable elastic loop pompom, a button would be sewn inside the hat. A crochet hook would be used to pull the two elastic ends into the hat near the button. Then the elastic ends could be tied (in a bow, or with a simple square knot... in a way that could be untied) securing them to the button.
With all of this in mind, lets look at a couple of faux fur pompoms that I bought when I first caught a bit-of-a-bug over pompom hats.
The smaller one is by Lion Brand. It came with an elastic cord -- but a rather long one. This makes it like my yarn pompom with added elastic. The loop could be pulled into a hat and tied to a button.
The larger one is by Bernat. This pompom has a heavy thread that could be used for attaching. If I desired to make it removable, I would replace the thread with a length of elastic using a large needle... 'Voila' another removable pompom.
*Last I checked the fur pompom with, the easy to use, small elastic loop was still available at https://www.amazon.com/dp/B0120RL2VQ/ref=twister_B0120R5U1E?_encoding=UTF8&psc=1