Assessment of ENSO and other variability in 104

[islasimpson]

@phillips-ad @swrneale here's a place to discuss ENSO skill and other aspects of variability in the new runs

[swrneale]

The easiest thing to do first is for me to add a ridiculous screenshot that @megandevlan and I looked at for the evolution of ENSO between 98 and 109. Bottom line it's complicated and data insufficient, but promising!

[swrneale]

But this is the 104 20th C and obe for the same period - pretty good.

[megandevlan]

Another metric, this is looking at the correlation of SST anomalies and the Nino 3.4 index within the Nino 3.4 box at various lag//lead times. It looks like 104 B1850 (peach dashed line) and the historical run (red dashed line) are two of the better performing simulations. Still showing somewhat of a too-regular ENSO pattern with elevated correlations well ahead of and after ENSO events, but certainly an improvement on other simulations.

[phillips-ad]

Looking at 104 in the CVDP, there were a couple metrics that gave me pause. First, the standard deviation of SST in the Tropical Pacific for DJF:

compared to that seen in 99 and obs:

I haven't seen such limited variability in the tropical Pacific SSTs in any prior CESM3 run. Here is a histogram of the nino3.4 standard deviations by month for 104LT/MT (observational estimates = orange lines):

compared to those from 92/98/99 (and earlier CESM3 runs):

[islasimpson]

I'm trying to reconcile this standard deviation plot with Rich's. Rich's is variance and this is standard deviation but it seems they are different. A standard deviation of 0.7 in January for the new runs in these plots would give a variance of 0.7 times 0.7 which is something more like 0.5 whereas RIch's variance plots are showing something more like 0.8. Maybe these are plotting different things aside from that?

[phillips-ad]

Two thoughts: Rich's is from the 104 historical, mine is from the piControl. And mine has been quadratically detrended prior to the calculation. There is a positive trend of .5->1C across the nino3.4 region in 104LT over the period I am looking at.

[swrneale]

A few things. Mine is just a linear de-trend, but unlikely to be the difference think. Also Adam is using 1979-2023 for the monthly std plots, when there is significant more variability than earlier in the century (granted that is pre-satellite) whereas I am using the whole period for the 104 HIST comparison. And yes there is no doubt the HIST simulations look more active than the 1850 for #104.
It is just really damn hard to say what the 'typical' observed behavior is because of the multi decadal variations in obs., and the model. Saying all that, I think the biggest problem that remains for CESM3 is the too long-lasting events.

[jfasullo]

I also have concerns about the CESM3 simulated patterns of ENSO teleconnections and attach a related figure below. The top two panels are CESM3 (98b). The mid left is GPCP, right is CESM2. The bottom ones are CESM3 biases versus each. Note the weakness of the NH ITCZ bias in the eastern Pacific and the strength in the SH. I see the same issue for radiative flues and across all members of CESM3. I suspect the issue is related to the equatorial dry zone bias.

[phillips-ad]

The nino3.4 autocorrelation looks better w/112 (upper right panel) compared to the 104 variants (2nd row), and looks more similar to the CESM2-piControl (bottom 2 rows):

The Tropical Pacific variability looks better in 112 as well:

The ENSO teleconnections are also slightly improved, and the Tropical Pacific dry slot has also improved slightly. Good news!
CVDP Comparison 104-112

[jfasullo]

Unfortunately I'm not seeing a big improvement in ENSO teleconnections in 112 (top right) - and still much lower than CESM2 (mid right). There is some noise in this metric, so its hard to be definitive, but the overall score I find of 0.78 (not shown) is still much less than CESM2 (0.84) for which the 2 sigma range is 0.02 based on CESM1-LE. See also the poor Z500* teleconnections attached.

[megandevlan]

It looks like the western extent of SST anomalies has grown in 112 as well (at least for the highest autocorrelation region):

The minimum variance of the monthly Nino3.4 index has increased a bit, but the maximum is still pretty low:

[megandevlan]

It looks like part of the discrepancy is the period looked at. I'm using years 1-90 and the Nino 3.4 autocorrelation looks fairly problematic:

But if I use years 6-50 as in @phillips-ad nice postage-stamp plot above, things aren't nearly as concerning.

[jfasullo]

Great point - for all metrics it helps to provide some range of uncertainty against the CESM2 LE when possible. ENSO in particular has considerable internal noise.

[phillips-ad]

Man, adding another 46 years to 112 definitely changed things. (Good catch on my analyzation years @megandevlan!) So I just extended my 112 data through year 96, and ran a CVDP comparison where I specified the last 80 years of 112 as the reference, 77 80-year slices of the CESM2 piControl, and 8 overlapping 80 year slices from observations. The nino3.4 autocorrelation is as previously showed of course, but I'm adding this as it shows the spread in CESM2 (top panel) and observations (bottom panel):

112 isn't really oscillating and is well outside the observational spread.

The ENSO spatial composites show very little response in the North Pacific with little deepening of the Aleutian Low (upper left plot is 112, the other 3 panels are random CESM2 slices):

That phenomenon may be linked to a weaker precipitation response. Here are difference plots for the ENSO precipitation spatial composite between 112 and CESM2 (top) and observations over land (bottom).

Looking at the top plot, CESM2 is much wetter (drier) in the Central Tropical Pacific (Western Tropical Pacific).

[swrneale]

I agree with @phillips-ad here definitely. CESM2 had about the right remote response with too strong nino3.4 SSTA amplitudes. CESM3 seems to have a better SSTA amplitude but with much weaker teleconnections. We need to look at the precip response more closely I imagine.

[islasimpson]

Given that in the past we had seen issues with the upper tropospheric westerlies in the Pacific sector, I had been meaning to check what that's currently looking like since it could impact ENSO teleconnections. Happy to report it looks good! No problems there.

[jfasullo]

FWIW, CMATv1 scores continue to be modest and certainly less than CESM2 for recent CESM3 members. Versions in the teens score notably worse. 121 seeks OK and within the uncertainty of internal variability from the best scores in earlier versions.

[phillips-ad]

CMATv1 scores continue to be modest and certainly less than CESM2 for recent CESM3 members.
This is very much true from a CVDP metrics perspective as well. It is primarily being driven by a drop in various ENSO metrics. Looking across the CMIP6 archive, CESM2 was one of the top performing models both for ENSO and non-ENSO CVDP metrics. (Here's an example comparison for combined historical/SSP370 ensembles.) Obviously CVDP metric scores are just one (rather simple) way to evaluate models. But if I compare recent CESM3 runs to the CMIP6 archive and look at CVDP metric scores, CESM3 is closer to the average CMIP6 model.

[swrneale]

Not sure if we're adding things here, but he is a quick look at the Winter Eastward and Summer Northward (Bay of Bengal) propagating. Looks pretty good and not a lot of sensitivity in previous runs. I might go for 115 or 121 as the best, but hard to call. And all are competitive with CESM2 (caveat first ens. member 1979-2005).

[megandevlan]

SST anomaly variance in the Nino 3.4 region is more reasonable in 121; the monthly max is still a little low, but better than 116/118.

But the period of ENSO looks to be really long in 121.

[dlawrenncar]

These are good and are the kinds of things we would like to review at the next CESM Project Meeting.

…

On Thu, Jan 9, 2025 at 9:24 AM Meg Fowler ***@***.***> wrote: SST anomaly variance in the Nino 3.4 region is more reasonable in 121; the monthly max is still a little low, but better than 116/118. image.png (view on web) <https://github.com/user-attachments/assets/4d733578-8399-4585-8b3e-d47454847517> But the period of ENSO looks to be *really* long in 121. Screenshot.2025-01-09.at.9.24.21.AM.png (view on web) <https://github.com/user-attachments/assets/5dd19943-4589-4201-9de7-d928fac69fef> — Reply to this email directly, view it on GitHub <#8 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFABYVEZUCIVEHWMF2U7MI32J2PFTAVCNFSM6AAAAABNYVKCE2VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCNZYHA3DMOI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[islasimpson]

Adam has the CVDP diagnostics here too: https://webext.cgd.ucar.edu/BLT1850/b.e30_beta04.BLT1850.ne30_t232_wgx3.121/cvdp/. I don't have any more to say about these at this point, beyond what Meg's already said above. ENSO variance is improved, there may be too much in the spring but time will tell, and period seems long.

[megandevlan]

Update using years 60-156 in run 121 - the period of ENSO is a bit better than for the initial 100 years, but it's still pretty long:

Obs, for comparison:

For context, I've plotted the number of months until the transition point on that red/blue autocorrelation plot. Run 121 is a bit high, but not too crazy relative to the rest of the development cycle.

[islasimpson]

@adamrher

[phillips-ad]

I have run two 121 time periods through the CVDP, years 20-156 and 60-156, with results here. For most of the plots, note the full 121 time period is in the upper right, while the shorter 60-156 time period is the first plot in the second row. The last 6 plots are time slices from the CESM2-picontrol. I do not see much of a difference at all between the 121 metrics for years 20-156 and those for years 60-156.

Overall, 121 seems to be a positive step forward. ENSO teleconnections, nino3.4 spectra and hovmollers, sst standard deviations in the tropics and the tropical pacific dry slot are all improved and/or near the best seen in the 1XX series of runs. 121's La Nina Hovmoller plot shows a particular improvement with capturing a 2nd year La Nina which is great to see.

Biases that still stand out: Still too weak of an ENSO teleconnection to the North Pacific, ENSO events last too long and the variability extends too far west as per @megandevlan's analysis, and 121 has a warming trend that is well outside what is seen in the CESM2 time slices. While 121 is an improvement over 104 and most 1XX simulations, overall CVDP scoring shows 121 to be worse than the 6 analyzed slices from the CESM2 picontrol.

[adamrher]

Hi Adam. It looks like to me, although perhaps it's too early to tell, that run 121 has transitioned to a more balanced equilibrium sometime after year 100, with RESTOM lower and global SSTs stabilizing. I'm wondering if the SST trends will be less in the second century.

[islasimpson]

Thanks for the diagnostics @phillips-ad. I put some of them in the slides for the project meeting tomorrow. Feel free to add anything else you think is important that I may have missed.

[phillips-ad]

Also in reference to the CAM7 meeting today where there were questions about when certain ENSO biases appeared in the development cycle: I created a slide for a CAS meeting back in November where I tried to look back on the timings of things:

[phillips-ad]

@islasimpson asked if I could run the CVDP on years 120-> on 121; results are here. I ran three time periods for 121 in this comparison: 120-200, 60-119 and 60-200. Many metrics look similar amongst those three time periods, though the nino3.4 autocorrelation does vary a bit, with years 120-200 (in upper right) going slightly longer:

The teleconnection to the North Pacific varies between those three periods as well; similar to how it varies between individual runs.

[islasimpson]

Thanks for this. We at least know that our problems don't go away with more equilibration of the run!

[JulioTBacmeister]

Thanks Adam. What are the last 2 plots? Are they runs 118 and 116? If so 118 doesn't look terrible - at any rate is very different form 121. Guess it's only 33 years.

[phillips-ad]

Yes, the last two plots are 118 and 116. The full nino3.4 autocorrelation plot that shows the above + 104, 109 and 112 as well as 5 CESM2 piControl time slices is here.

[islasimpson]

A group of us discussed yesterday the possibility of using initialized predictions of 1997/1998 to explore the issue of the long decorrelation timescale of Nino3.4. I'm thinking we should maybe pause on this plan. I've been looking at how this particular metric evolves over the 121 historical simulation. Firstly, below is a comparison of the Nino3.4 autocorrelation between the 121 simulations and the historical simulations from CMIP6 and LENS2. For observations I'm using 1854-2024, for LENS2 I'm using 1920-2024, for the 121 simulations I'm using the full length of the simulation. There are indications from this that the historical is closer to the observations than the piControl.

In the process of making this plot, I noticed that the historical got even closer to the observations if I used a more recent time period. So, what I'm plotting below is the month at which the lagged autocorrelation crosses zero for 50 year running segments for the 121 simulations and observations. From this we can see that the nino3.4 autocorrelation systematically evolves over the historical simulation to become more like observations. Observations don't show that same evolution, so I don't think this is a real global warming signal that we should be having in the model. But something over the historical simulation is kicking it into being more like observations. At some point, someone mentioned that maybe it's a problem resulting from the model being too cold. Seems like this would fit with that hypothesis. Also @gustavo-marques I wonder if it's worth checking how the salinity bias you mentioned is evolving over the historical simulation.

But overall, I'm not convinced that an initialized 1997/1998 prediction is going to help us with this problem now.

[islasimpson]

I just realized the above has no legend. Blue is 121 piControl, red is 121 historical, black is obs.

[islasimpson]

Here's a version of the above figure but now with LENS2 added (and a legend)

[gustavo-marques]

Salinity

Here is a time progression of the salinity structure across three longitudes at the Equator. The left panels are 121 BHIST, and the right are obs-based estimates. Averages are between 1850-1881, 1901-1932, 1944-1975, and 1975-2006. There is not much change throughout the simulation.

West Pacific (Lon = 143)

Central Pacific (Lon = 190)

East Pacific (Lon = 265)

Potential Temperature

Similarly, here is a time progression of the potential temperature structure across the same longitudes and periods.
The ocean has been warming near the surface since ~ 1940, becoming more similar to obs.

West Pacific (Lon = 143)

Central Pacific (Lon = 190)

East Pacific (Lon = 265)

[adamrher]

At some point, someone mentioned that maybe it's a problem resulting from the model being too cold. Seems like this would fit with that hypothesis

I agree with @islasimpson here.

Separately, I wanted to share this result from GFDL, who are in the process tuning up ENSO in CM5. Below shows that reducing the entrainment rate in their deep convection scheme (rkm_dp1 = 0.9->0.5) increases the amplitude of the teleconnections (regression of Nino3.4 on Z500). This may be something to consider.

[megandevlan]

Adding onto @islasimpson's suggestion that it's the cold bias driving too long of an ENSO period, there's some support in the 1850 runs for this (though I've excluded run 91 as an outlier) - scatter plot of mean equatorial Pacific SST vs. month of transition in the autocorrelation plots:

[megandevlan]

It would be nice to do this for pieces of the 121_HIST run too, but didn't quite get there today; will prioritize for next week!

[islasimpson]

Some more indication that warming up is what we need to do. Added 4xCO2 to the plot...

@megandevlan - it does seems like it would be nice to consider scatter plots like the one you made above but for the historical, 4xCO2, and 1%CO2 segments.

[islasimpson]

Maybe also with CESM2 as it's not actually clear that 121 is that much colder than CESM2 from this comparison from @phillips-ad if temperatures for 1940-2012.

[phillips-ad]

I split 121's BLTHIST run into three time periods (1850-1939, 1940-2012 and 1850-2012) and compared them against CESM2-LENS in a new CVDP comparison.

Notable results:
(As a cropped version of each metric is shown below I included links to the full images.)

• The ENSO teleconnection to the North Pacific is much improved in the 1940-2012 time frame looking at ENSO spatial composites (see upper right panel compared to observations over the same time period in the upper left). Also note how much weaker the same teleconnection is from 1850-1939 in the lower left panel.
  

• The nino3.4 spectra for the 1940-2012 time period continues to match observations well.
  

• The nino3.4 monthly standard deviation still compares well to obs (orange lines).
  

• The El Nino Hovmoller for 1940-2012 shows a transition to La Nina (upper right panel ) which better matches obs (upper left), contrary to the 1850-1939 panel (lower left):
  

[megandevlan]

Adding onto @phillips-ad comments above, I used years 1940-2012 for 121_hist in the plots below. 121_4xco2 is cut to be years 50-152.

• The monthly standard deviations across the 121 simulations are pretty consistent, and not terrible. It would be nice to increase the monthly maximum, but we've been low since ~104.

• The autocorrelation is better in 121_hist than 121 (as @islasimpson's noted). It's looking good in 1pctco2 and in 4xCO2 too:

• I need to update this plot to include CESM2, but looking across the 121 series (red), it does appear that there's a link between mean SSTs getting warmer and a decrease in the autocorrelation transition time; the two overlapping labels are 121_hist (left point) and 121_1pctco2 (right point)

• The western extent in hist vs. the pre-ind case is a little worse, but not necessarily dramatically

[megandevlan]

Adding CESM2 and CESM1 for reference; it does seem like CESM2 is generally warmer than the CESM3 dev runs

[swrneale]

These a really interesting. Given Adam's composite hovmollers and that most of the lack of transition to La Nina is away from the East Pacific, I'm wondering @megandevlan could you repeat the autocorrelation, but for the nino4, nino3 and even nino1+2 regions separately?
Also given the question regarding the El Nino remote response from the AMWG mtg, @cecilehannay could we possibly run a full historically FLTHIST simulation?

[megandevlan]

I don't have the 1+2 region yet, but I can add nino3 and nino4 pretty easily. We've generally looked better for the Nino4 region

[swrneale]

Hmm, I was thinking this might look the opposite (i.e., further west (nino4) would have a longer lag crossover) since in the East Pacific the SSTAs crossover to cold values a bit more reliably. It's important to remember I guess, that the lag-correltion plots reflect Nina events the same as nino events.

[islasimpson]

Here's where we were at in terms of Nino3.4 autocorrelation before the lab sea freeze took over. Run 129 was looking like it might have resolved our ENSO autocorrelation problem but it was also freezing over so we weren't sure.

This is also showing ENSO variance. A doubled peaked structure in longitude has been argued by Pedro DiNezio and Brandon Molina to accompany a lack of oscillatory behavior in ENSO. Not totally confident that holds in these runs, but of all the ones below, 129 has less of a secondary peak to the west around the dateline

129 was also looking pretty good in El Nino hovmollers

There were concerns that it may not be getting the tendency for two-year La Nina...

ENSO teleconnections may have been too weak in the latest runs too. But these were relatively short and also freezing. Just something to look out for in future runs.

[jfasullo]

In 156/160/162 I continue to get poor scores in CMAT, mainly related to low ENSO teleconnections scores. The attached plot shows the Jul-Jun teleconnection analysis for 156. The biases against ERA5 are shown in the bottom left and resemble in many ways the changes from CESM2 in the bottom right - and thus the model is quite biased and also worse than before, with excessive zonal extent in both the eastern and western Pacific. Extratropical responses suffer and exhibit a westward displacement.

[swrneale]

It would be good to understand this along slide the CVDP diags. here https://docs.google.com/presentation/d/1rTfw3zglYzJUe8JQHYfsd5mDU5XYFPd7eX0mzlwsGhs/edit?slide=id.g3567c083acb_0_20#slide=id.g3567c083acb_0_20
Slides 28/29. I still think we have a failure of the Kelvin wave induced cooling in the East Pacific that terminates the struggling La Nina, and so El Nino persists into the Summer, messing up your teleconnections. We need help with the response in MOM which Gustavo has started to do, but he's out for a while.

[phillips-ad]

@jfasullo it looks like you are looking at 30 years of 156 is that correct? If so I am guessing the shorter time-slice (vs. 50 year slices shown in the CVDP) explains the difference.

[phillips-ad]

I have created a CVDP comparison using the following simulations and time slices:

• 163 (years 25-76)
• 162 (1-50, 51-100, 25-100)
• 156 (91-140, 141-190, 191-240, 80-240)
• 121 (101-150, and 151-200)
• CESM2 piC (3 50 year timeslices)

The comparison is here:
https://webext.cgd.ucar.edu/BLT1850/b.e30_alpha06e.B1850C_LTso.ne30_t232_wgx3.163/cvdp/index.html
(Note that the years analyzed for 162 and 156 are always in the panel title.)

A few items are notable:

• 162 and 156 stand out in the (ENSO dominated) CVDP metrics as being on par with the CESM2 piControl segments, both for pattern correlations and RMS. It's been a while since that's been said of a CESM3 simulation.
• 156/162 look pretty good for whatever time slice was analyzed with regards to nino3.4 specta and ENSO teleconnections in NH winter and spring. Springtime specifically has historically been not-so-good for CESM.
• The springtime ENSO teleconnection improvement though might have to do with the nino3.4 seasonal cycle being off.
• 156/162 are more of a mixed bag for El Nino and La Nina Hovmollers.
• Westward extent of Tropical Pacific SST variability continues to be a problem in CESM3 simulations (compared to observations and CESM2).

[megandevlan]

These are really great to look through! I wonder if the mean SST bias pattern might be leading to the odd seasonal cycle in 162? The warm bias around the ITCZ stands out compared to other sims there, but this isn't a plot I look at often! Do you have a sense of whether this is anomalous for most CESM3 simulations?

[swrneale]

Are the obs. 1850 SSTs?

[phillips-ad]

Yah @swrneale nailed it. The SST's are being compared to 1950-2024 ERSSTv5. And the Tropical Pacific cold bias region (which might not be so cold when compared to 1850's SSTs) is also seemingly sitting right under CESM's usual dry slot bias. I think @islasimpson has brought this up before.

[jfasullo]

Looking at ENSO teleconnections to precipitation in 170-2 I found them to very poor - extending zonally across the Indo-Pacific (attached). Teleconnections in 180 are improved, yet the precipitation teleconnections don’t look quite as good in 178/9 as 180 (attached), though still improved from 170-2 at about r=0.4. Pattern correlations are 0.48, 0.51, and 0.63 respectively, differences that may (but probably don’t) arise from internal variability. The strength over the maritime continent suggests an anchoring by the warm SST biases? The dry
slot in 179 is particularly concerning but present in all members to some extent. Summary table also attached.

[jfasullo]

I see a fairly dramatic difference in ENSO teleconnections between 234pi and 235pi - with 234 being the better performer. See for example the ENSO PSL teleconnections in CMAT

[jfasullo]

CMAT scores continue to improve in CESM3 and 237 is one of the best runs in a while based on these (within the noise of 234).

[islasimpson]

I think this is consistent with 235 having much weaker ENSO variance. Pasting a slide here from the project meeting for the records. It sounds like 235 is not viable from the oceanographers perspective. It'll be interesting to see how things evolve in 236 when it's running.

[jfasullo]

The most recent versions of CESM3 continue to score well in CMAT. #245 shows some of the best overall scores on record.

[jfasullo]

The newer versions of the model continue to score well in CMAT. Here a number of runs are sorted by overall score.