Can any harm occur by simply enabling jumbo frames on your switch infrastructure?
Depends on your definition of harm. On modern switches, there's no impact usually. But it doesn't do anything either unless some nodes actually begin using jumbos.
I understand that in a given flow from one machine to another all participants need to support jumbo frames in order for them to arrive intact.
Hosts are a different thing. They are the nodes that create packets/frames, so they actually decide which frame sizes reach a switch.
For jumbo frames to work, all nodes (switches, routers, hosts) on a given subnet/VLAN need to support the same frame size - there's no mechanism for negotiating that size. Of course, sending oversized IPv4 packets inside jumbo frames across a router forces that router to fragment the packets - that should really be avoided.
However, by default, all the switches I work with have jumbos disabled in favor of the standard 1500 frame size.
Yes, pretty much all managed switches use the standard maximum frame size by default.
What, if any, harm can occur if one simply enables jumbo everywhere (even if it's not being used yet)?
Potential problems include (most of which I've actually seen):
- incompatibilities with nodes claiming to but ultimately being unable to process jumbo frames
- unequal frame sizes used across the network (IP subnet/VLAN) due to definition blur (maximum frame size with or without 802.1Q tagging, "MTU" for max L2 frame size, etc.)
- nodes may switch from hardware processing to software processing -> increase in CPU load and decrease in performance
- further performance hits due to inadequate testing by the vendor or support in the hardware
- excessive frame drops
- latency increases across the network, especially with QoS
And the obvious follow-up question: if there is no harm, why are jumbos not the default?
By default anyone uses the standard. IEEE 802.3 doesn't recognize frames with payloads larger than 1500 bytes.
The general rule is to limit jumbos to well-defined zones (e.g. SAN) and test your equipment well.
The original purpose of jumbos - increase in throughput by decrease in processing overhead - is pretty much gone, supplanted by a broad increase in hardware processing and offloading. What's left is a mere increase in line efficiency by 4.4%[*1] - not really worth the trouble.
The only actual use of jumbos is when you're tunneling: inflating the outer frame size saves you from reducing the inner MTU. Sadly, most scenarios where this would be handy don't allow for jumbo frames.
[*1] For TCP over IPv4 over Ethernet without options, 1460 bytes of application data can be transported by a maximum frame of 1538 bytes on the wire (including inter-frame gap), resulting in an line efficiency of 94.93%. 9k jumbos can transport 8960 application bytes with 9038 bytes on the wire => 99.14%. (8960/9038)/(1460/1538) shows +4.43% of efficiency increase and potentially increased throughput.
