Well snowmobiles and dirt bikes both come in a variety of sizes, different sizes for different purposes.
A dual-sport bike is a compromise, meant to perform well on pavement or dirt, not specifically designed for either. As you might expect, some are more for the road, meant to be taken off road only occasionally, and some are mostly for the dirt, just barely road-legal.
So it depends on how you see yourself riding. If you're just riding on dirt you will very seldom go faster than about 30 mph, so a 250 is plenty big enough, especially because you want something light and maneuverable. If you plan to use it mostly on the road, and you need to ride on highways at 60mph+ for long distances, then 650cc is a better size. A 250 can't cruise at highway speeds, you need at least 500cc. Of course a bike that big is not optimum on dirt.
Dirt bikes have long suspension travel, so they tend to be very tall, the seat high off the ground. So they're not good for the 'inseam challenged', because you want to be able to sit on a bike with both feet flat on the ground. And this is especially important if you're a beginner. I'm 6' and I've sat on dual sports that even I couldn't 'flatfoot'!