Rumbling in a water heater indicates an issue with the buildup of sediment or mineral deposits. This sediment can accumulate at the bottom of the tank and overheat, causing loud noises. Sediment buildup can also reduce the efficiency of the water heater, leading to increased energy consumption. Flushing the water heater regularly and replacing the anode rod can help prevent sediment buildup and prolong the life of the unit.