The short answer is, "Of course you can." The device is not "locked" or otherwise hobbled to prevent you from doing so. (At least, I don't think it is.)
The longer answer is, "Of course you can, but it won't be easy." WinCE is an OS that was intended for use by OEMs in embedded devices, not a consumer-level OS like Windows. It's also obsolete. That means you're not likely to find a large, easy to navigate ecosystem of WinCE tools, drivers, apps, and folks available to offer help in support of your project like you might if you were working with a modern and well-supported and/or open source platform.
To change the way the device boots, you'd need to flash new firmware with the OS and device-specific drivers, set to launch the WinCE app(s) at bootup or a user interface to present you with those apps.
You're likely to find that the effort involved to figure out how to do all that is not worth it.