What in the world is “a proprietary OS I cannot trust”. What’s your actual threat model? Have you actually run any risk analyses or code audits against these OSes vs. (i assume) Linux to know for sure that you can trust any give FOSS OS? You do realize there’s still an OS on your dumb switch, right?
This is a silly reason to not learn to manage your networking hardware.
A VLAN is (theoretically) equivalent to a physically separated layer 2 domain. The only way for machines to communicate between vlans is via a gateway interface.
If you don’t trust the operating system, then you don’t trust that it won’t change it’s IP/subnet to just hop onto the other network. Or even send packets with the other network’s header and spoof packets onto the other subnets.
It’s trivially easy to malform broadcast traffic and hop subnets, or to use various arp table attacks to trick the switching device. If you need to segregate traffic, you need a VLAN.
Edit: Should probably note that simply VLAN tagging from the endpoints on a trunk port isn’t any better than subnetting, since an untrusted machine can just tag packets however it wants. You need to use an 802.1q aware switch and gateway to use VLANs effectively.
What you are asking will work. That’s the whole point of subnets. No you don’t need a VLAN to segregate traffic. It can be helpful for things like broadcast control.
However, you used the word “trust” which means that this is a security concern. If you are subnetting because of trust, then yes you absolutely do need to use VLANs.
Broadcast traffic (such as DHCP) doesn’t cross subnets without a router configured to forward it. It’s one of the reasons subnets exist.