Wrap the pencil with teflon tape, as Andy said... then instead of glue, paint the chain with clear enamel or acrylic and let dry... maybe 2-3 coats total, for a bit of added rigidity.
The clear will act as a "glue," and the teflon tape will slide right off. The wheel won't be as strong as if you had used CA glue, so be careful when you handle it.