The majority of my homelab consists of two servers: A Proxmox hypervisor and a TrueNAS file server. The bulk of my LAN traffic is between these two servers. At the moment, both servers are on my “main” VLAN. I have separate VLANs for guests and IoT devices, but everything else lives on VLAN2.
I have been considering the idea of creating another VLAN for storage, but I’m debating if there is any benefit to this. My NAS still needs to be accessible to non-VLAN-aware devices (my desktop PC, for instance), so from a security standpoint, there’s not much benefit; it wouldn’t be isolated. Both servers have a 10Gb DAC back to the switch, so bandwidth isn’t really a factor; even if it was, my switch is still only going to switch packets between the two servers; it’s not like it’s flooding the rest of my network.
Having a VLAN for storage seems like it’s the “best practice,” but since both servers still need to be accessible outside the VLAN, the only benefit I can see is limiting broadcast traffic, and as far as I know (correct me if I’m wrong), SMB/NFS/iSCSI are all unicast.
iSCSI is block level storage where NFS/SMB are file level… When you browse a folder with SMB/NFS it is going to ask the remote service for the meta file list then cache the whole thing till it thinks it needs it refreshed… iSCSI is going to go read a set of blocks to read the metadata off the remote file system. iSCSI can be considerably more chatty between the two hosts as it is lower level.