EEE PC (with Linux) that we bought for travels. It came with a Tux Paint software that our daughter, then 3, mastered quickly. It was and still is a big life saver during long flights.
The program has an easy kid-friendly but not too-cartoony interface, and a very good set of tools (brushes, stamps, "magic" effects, etc) and some coloring pages. It's all your child needs and even more.
It supports various operation systems, so it will run on whatever you have. And you know what the best part is? It's FREE! You can download it from their website: tuxpaint.org
We also got some "painting" apps for iPod. They are not exactly drawing apps, but are fun nevertheless.
Doodle Kids -this is good for small ones, there are no tools or controls, everything is random. All you do is doodle. But doodling is good too, right?
Make a Martian - this is an alien making "factory". It's fun, cute, simple, and makes my daughter giggle every time she plays with it.
If you like an educational twist, the Dot-to-Dot Number Whiz is a good one. The level of difficulty can be adjusted. So your child can practice 123s or ABCs, or skip counting.
Let me know in the comments if you know about any other good paint/doodle apps for small kids.
(photo by mamalya, snapshot of the Martian we made)