I believe that kelp actually thrives in colder temps. My guess would be swell + rain potential and the likelihood of encountering less than ideal diving conditions at near shore sites.
Most people don't realize this, but San Diego has some of the best winter breaks in California (having visions of perfect widow peaks at Black's). During our winter in the northern hemisphere, the primary source of swell comes from the cyclonic activity that moves across the northern pacific, from west to east. When these large systems reach our shores, they bring long period NW ground swell energy as well as some precipitation.
While most of So Cal is somewhat shadowed from the steep angled NW swell (due to offshore islands and shape of coast), once you get south of Oceanside (and the shadow of San Clemente Island), the swell window of the coast line has a direct view to the W/NW. Coupled with those deep underwater canyons that funnel the ground swell energy into some select focal points around south county and magic things happen. Good for surfing ... not so good for diving.
