Yes! Which parts of the garage to insulate will depend on whether it is attached to the house or not, and how much you plan to use it, but the benefits of insulating the garage ultimately outweigh the small initial cost required to insulate it.
If the garage is attached to the home, then insulating the internal wall will help prevent heat loss from the home and reduce heating costs. The savings in heating costs would easy recoup the cost of the insulation within a few years. Most heat in the garage is lost through the attic space, so insulating the roof of the garage would also be worthwhile.
So, it is worth insulating a free-standing garage? It depends on how you use it. If you plan on spending time in the garage, either as a living space or as a workshop, then insulating it will make it warmer in the winter and cooler in the summer, making it far more comfortable to spend time in. For maximum results, insulating the walls, roof and door is essential. Paired with a garage door threshold seal, the garage will be temperature controlled and free from water ingress, drafts, dust and vermin.