Skip to content

Commit 3f1032b

Browse files
mbercxmbercx
committed
✨ PwOutput: gate total_energy on the calculation type
For `nscf` and `bands` runs QE never assigns `etot` and writes `<etot>0.0</etot>` to the XML. When such a run shares its `prefix` with a prior `scf` run — the standard `scf → nscf → bands` workflow — the nscf run also overwrites the scf XML at `<prefix>.save/`, so the meaningful total energy is gone from disk regardless of what the user asked for. `total_energy` is now resolved with a `Coalesce` of two branches: an XML branch gated by `Check` on `xml.input.control_variables.calculation`, and a stdout fallback. The XML branch is taken only for documented calculations types (scf, relax, vc-relax, md, vc-md). For nscf/bands runs the gate fails and `Coalesce` falls through to the stdout branch. `PwStdoutParser` now extracts the converged energy from the last `! total energy = ... Ry` line (relax/md print one per ionic step; the final value is the converged one). Side effect: the `collinear` fixture (which is itself an nscf XML) previously surfaced `total_energy: 0.0`. With the gate in place the field is correctly absent — the `0.0` was always meaningless. The regression snapshot drops it.
1 parent 4e4885f commit 3f1032b

11 files changed

Lines changed: 300 additions & 5 deletions

File tree

‎src/qe_tools/outputs/parsers/pw.py‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,7 @@ def parse(content):
5858
r"([\-\d.E+]+)\s+([\-\d.E+]+)"
5959
)
6060
_HOMO_RE = re.compile(r"highest occupied level\s*\(ev\):\s*([\-\d.E+]+)")
61+
_TOTAL_ENERGY_RE = re.compile(r"!\s+total energy\s*=\s*([\-\d.E+]+)\s*Ry")
6162

6263

6364
class PwStdoutParser(BaseStdoutParser):
@@ -78,4 +79,10 @@ def parse(content: str) -> dict:
7879
if match:
7980
parsed_data["highest_occupied_level"] = float(match.group(1))
8081

82+
# Final SCF total energy in Ry. For relax/md runs QE prints one `!` line
83+
# per ionic step; the final converged value is the last one.
84+
energy_matches = _TOTAL_ENERGY_RE.findall(content)
85+
if energy_matches:
86+
parsed_data["total_energy"] = float(energy_matches[-1])
87+
8188
return parsed_data

‎src/qe_tools/outputs/pw.py‎

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
from typing import Annotated, TextIO
77

88
import numpy as np
9-
from glom import Coalesce, Spec
9+
from glom import Check, Coalesce, Spec
1010

1111
from dough import Unit
1212
from dough.converters import BaseConverter
@@ -324,9 +324,19 @@ class _PwMapping:
324324
total_energy: Annotated[
325325
float,
326326
Spec(
327-
(
328-
"xml.output.total_energy.etot",
329-
lambda energy: energy * CONSTANTS.hartree_to_ev,
327+
Coalesce(
328+
(
329+
# For `nscf` and `bands` QE never assigns `etot` and writes
330+
# `<etot>0.0</etot>` (see PW/src/non_scf.f90 and
331+
# PW/src/pw_restart_new.f90); fall through to stdout below.
332+
Check(
333+
"xml.input.control_variables.calculation",
334+
one_of=("scf", "relax", "vc-relax", "md", "vc-md"),
335+
),
336+
"xml.output.total_energy.etot",
337+
lambda energy: energy * CONSTANTS.hartree_to_ev,
338+
),
339+
("stdout.total_energy", lambda energy: energy * CONSTANTS.ry_to_ev),
330340
)
331341
),
332342
Unit("eV"),

‎tests/outputs/fixtures/pw/nscf_etot_clobber/data-file-schema.xml‎

Lines changed: 236 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,236 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<qes:espresso xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:qes="http://www.quantum-espresso.org/ns/qes/qes-1.0" xsi:schemaLocation="http://www.quantum-espresso.org/ns/qes/qes-1.0 http://www.quantum-espresso.org/ns/qes/qes_250521.xsd" Units="Hartree atomic units">
3+
<!-- All quantities are in Hartree atomic units unless otherwise specified -->
4+
<general_info>
5+
<xml_format NAME="QEXSD" VERSION="25.05.21">QEXSD_25.05.21</xml_format>
6+
<creator NAME="PWSCF" VERSION="7.5">XML file generated by PWSCF</creator>
7+
<created DATE=" 7May2026" TIME="16:24: 5">This run was terminated on: 16:24: 5 7 May 2026</created>
8+
<job></job>
9+
</general_info>
10+
<parallel_info>
11+
<nprocs>1</nprocs>
12+
<nthreads>1</nthreads>
13+
<ntasks>1</ntasks>
14+
<nbgrp>1</nbgrp>
15+
<npool>1</npool>
16+
<ndiag>1</ndiag>
17+
</parallel_info>
18+
<input>
19+
<control_variables>
20+
<title></title>
21+
<calculation>nscf</calculation>
22+
<restart_mode>from_scratch</restart_mode>
23+
<prefix>MgO</prefix>
24+
<pseudo_dir>../PP</pseudo_dir>
25+
<outdir>../tmp</outdir>
26+
<stress>false</stress>
27+
<forces>false</forces>
28+
<wf_collect>true</wf_collect>
29+
<disk_io>low</disk_io>
30+
<max_seconds>10000000</max_seconds>
31+
<nstep>1</nstep>
32+
<etot_conv_thr>5.000000000000000E-005</etot_conv_thr>
33+
<forc_conv_thr>5.000000000000000E-004</forc_conv_thr>
34+
<press_conv_thr>5.000000000000000E-001</press_conv_thr>
35+
<verbosity>low</verbosity>
36+
<print_every>100000</print_every>
37+
<fcp>false</fcp>
38+
<rism>false</rism>
39+
</control_variables>
40+
<atomic_species ntyp="2">
41+
<species name="Mg">
42+
<mass>2.403500000000000E+001</mass>
43+
<pseudo_file>Mg.pbe-n-kjpaw_psl.0.3.0.UPF</pseudo_file>
44+
</species>
45+
<species name="O">
46+
<mass>1.599940000000000E+001</mass>
47+
<pseudo_file>O.pbe-n-kjpaw_psl.0.1.UPF</pseudo_file>
48+
</species>
49+
</atomic_species>
50+
<atomic_structure nat="2" alat="8.0374557182000004" bravais_index="2">
51+
<atomic_positions>
52+
<atom name="Mg" index="1">0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000</atom>
53+
<atom name="O" index="2">4.018727859100000E+000 4.018727859100000E+000 4.018727859100000E+000</atom>
54+
</atomic_positions>
55+
<cell>
56+
<a1>-4.018727859100000E+000 0.000000000000000E+000 4.018727859100000E+000</a1>
57+
<a2>0.000000000000000E+000 4.018727859100000E+000 4.018727859100000E+000</a2>
58+
<a3>-4.018727859100000E+000 4.018727859100000E+000 0.000000000000000E+000</a3>
59+
</cell>
60+
</atomic_structure>
61+
<dft>
62+
<functional>PBE</functional>
63+
</dft>
64+
<spin>
65+
<lsda>false</lsda>
66+
<noncolin>false</noncolin>
67+
<spinorbit>false</spinorbit>
68+
</spin>
69+
<bands>
70+
<nbnd>8</nbnd>
71+
<tot_charge>0.000000000000000E+000</tot_charge>
72+
<occupations>tetrahedra</occupations>
73+
</bands>
74+
<basis>
75+
<gamma_only>false</gamma_only>
76+
<ecutwfc>2.500000000000000E+001</ecutwfc>
77+
<ecutrho>2.000000000000000E+002</ecutrho>
78+
</basis>
79+
<electron_control>
80+
<diagonalization>davidson</diagonalization>
81+
<mixing_mode>plain</mixing_mode>
82+
<mixing_beta>7.000000000000000E-001</mixing_beta>
83+
<conv_thr>5.000000000000000E-007</conv_thr>
84+
<mixing_ndim>8</mixing_ndim>
85+
<max_nstep>100</max_nstep>
86+
<exx_nstep>100</exx_nstep>
87+
<real_space_q>false</real_space_q>
88+
<real_space_beta>false</real_space_beta>
89+
<tq_smoothing>false</tq_smoothing>
90+
<tbeta_smoothing>false</tbeta_smoothing>
91+
<diago_thr_init>0.000000000000000E+000</diago_thr_init>
92+
<diago_full_acc>false</diago_full_acc>
93+
<diago_cg_maxiter>20</diago_cg_maxiter>
94+
<diago_rmm_ndim>4</diago_rmm_ndim>
95+
<diago_gs_nblock>16</diago_gs_nblock>
96+
<diago_rmm_conv>false</diago_rmm_conv>
97+
</electron_control>
98+
<k_points_IBZ>
99+
<monkhorst_pack nk1="16" nk2="16" nk3="16" k1="0" k2="0" k3="0">Monkhorst-Pack</monkhorst_pack>
100+
</k_points_IBZ>
101+
<ion_control>
102+
<ion_dynamics>none</ion_dynamics>
103+
<upscale>1.000000000000000E+002</upscale>
104+
<remove_rigid_rot>false</remove_rigid_rot>
105+
<refold_pos>false</refold_pos>
106+
</ion_control>
107+
<cell_control>
108+
<cell_dynamics>none</cell_dynamics>
109+
<pressure>0.000000000000000E+000</pressure>
110+
<wmass>0.000000000000000E+000</wmass>
111+
<cell_do_free>all</cell_do_free>
112+
</cell_control>
113+
<symmetry_flags>
114+
<nosym>false</nosym>
115+
<nosym_evc>false</nosym_evc>
116+
<noinv>false</noinv>
117+
<no_t_rev>false</no_t_rev>
118+
<force_symmorphic>false</force_symmorphic>
119+
<use_all_frac>false</use_all_frac>
120+
</symmetry_flags>
121+
<free_positions rank="2" dims=" 3 2">
122+
1 1 1
123+
1 1 1
124+
</free_positions>
125+
<twoch_>
126+
<twochem>false</twochem>
127+
<nbnd_cond>0</nbnd_cond>
128+
<degauss_cond>0.000000000000000E+000</degauss_cond>
129+
<nelec_cond>0.000000000000000E+000</nelec_cond>
130+
</twoch_>
131+
</input>
132+
<output>
133+
<convergence_info>
134+
<scf_conv>
135+
<convergence_achieved>false</convergence_achieved>
136+
<n_scf_steps>1</n_scf_steps>
137+
<scf_error>0.000000000000000E+000</scf_error>
138+
</scf_conv>
139+
<wf_collected>true</wf_collected>
140+
</convergence_info>
141+
<algorithmic_info>
142+
<real_space_q>false</real_space_q>
143+
<real_space_beta>false</real_space_beta>
144+
<uspp>true</uspp>
145+
<paw>true</paw>
146+
</algorithmic_info>
147+
<atomic_species ntyp="2" pseudo_dir="../PP/">
148+
<species name="Mg">
149+
<mass>2.403500000000000E+001</mass>
150+
<pseudo_file>Mg.pbe-n-kjpaw_psl.0.3.0.UPF</pseudo_file>
151+
</species>
152+
<species name="O">
153+
<mass>1.599940000000000E+001</mass>
154+
<pseudo_file>O.pbe-n-kjpaw_psl.0.1.UPF</pseudo_file>
155+
</species>
156+
</atomic_species>
157+
<atomic_structure nat="2" num_of_atomic_wfc="8" alat="8.0374557182000004" bravais_index="2">
158+
<atomic_positions>
159+
<atom name="Mg" index="1">
160+
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
161+
</atom>
162+
<atom name="O" index="2">4.018727859100000E+000 4.018727859100000E+000 4.018727859100000E+000</atom>
163+
</atomic_positions>
164+
<cell>
165+
<a1>-4.018727859100000E+000 0.000000000000000E+000 4.018727859100000E+000</a1>
166+
<a2>0.000000000000000E+000 4.018727859100000E+000 4.018727859100000E+000</a2>
167+
<a3>-4.018727859100000E+000 4.018727859100000E+000 0.000000000000000E+000</a3>
168+
</cell>
169+
</atomic_structure>
170+
<basis_set>
171+
<gamma_only>false</gamma_only>
172+
<ecutwfc>2.500000000000000E+001</ecutwfc>
173+
<ecutrho>2.000000000000000E+002</ecutrho>
174+
<fft_grid nr1="40" nr2="40" nr3="40"></fft_grid>
175+
<fft_smooth nr1="25" nr2="25" nr3="25"></fft_smooth>
176+
<fft_box nr1="40" nr2="40" nr3="40"></fft_box>
177+
<ngm>17477</ngm>
178+
<ngms>6183</ngms>
179+
<npwx>790</npwx>
180+
<reciprocal_lattice>
181+
<b1>
182+
-1.000000000000000E+000 -1.000000000000000E+000 1.000000000000000E+000
183+
</b1>
184+
<b2>1.000000000000000E+000 1.000000000000000E+000 1.000000000000000E+000</b2>
185+
<b3>-1.000000000000000E+000 1.000000000000000E+000 -1.000000000000000E+000</b3>
186+
</reciprocal_lattice>
187+
</basis_set>
188+
<dft>
189+
<functional>PBE</functional>
190+
</dft>
191+
<magnetization>
192+
<lsda>false</lsda>
193+
<noncolin>false</noncolin>
194+
<spinorbit>false</spinorbit>
195+
<absolute>0.000000000000000E+000</absolute>
196+
</magnetization>
197+
<total_energy>
198+
<etot>0.000000000000000E+000</etot>
199+
<eband>-2.412528325153409E-002</eband>
200+
<ehart>5.173955844516139E+000</ehart>
201+
<vtxc>-5.381547635132835E+000</vtxc>
202+
<etxc>-4.655081273351107E+000</etxc>
203+
<ewald>0.000000000000000E+000</ewald>
204+
</total_energy>
205+
<band_structure>
206+
<lsda>false</lsda>
207+
<noncolin>false</noncolin>
208+
<spinorbit>false</spinorbit>
209+
<nbnd>8</nbnd>
210+
<nelec>8.000000000000000E+000</nelec>
211+
<fermi_energy>2.208720474420502E-001</fermi_energy>
212+
<starting_k_points>
213+
<monkhorst_pack nk1="16" nk2="16" nk3="16" k1="0" k2="0" k3="0">Monkhorst-Pack</monkhorst_pack>
214+
</starting_k_points>
215+
<nks>145</nks>
216+
<occupations_kind>tetrahedra</occupations_kind>
217+
<ks_energies>
218+
<k_point weight="4.8828125000000000E-004">0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000</k_point>
219+
<npw>749</npw>
220+
<eigenvalues size="8">
221+
-4.333761183969641E-001 1.964059244895351E-001 1.964059244907791E-001
222+
1.964059244947730E-001 3.597282634048695E-001
223+
7.717057859626861E-001 7.717057860499605E-001 7.717057861123908E-001
224+
</eigenvalues>
225+
<occupations size="8">
226+
9.999999999999996E-001 9.999999999999996E-001 9.999999999999996E-001
227+
9.999999999999996E-001 0.000000000000000E+000
228+
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
229+
</occupations>
230+
</ks_energies>
231+
</band_structure>
232+
</output>
233+
<exit_status>0</exit_status>
234+
<cputime>0</cputime>
235+
<closed DATE=" 7 May 2026" TIME="16:24: 5"></closed>
236+
</qes:espresso>

‎tests/outputs/fixtures/pw/nscf_etot_clobber/scf.out‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
Program PWSCF v.7.5 starts on 7May2026 at 16:23:36
2+
3+
--- Removed: SCF cycle, eigenvalues, k-points; only keep header + final energies + JOB DONE. ---
4+
5+
highest occupied level (ev): 5.3452
6+
7+
! total energy = -75.53725762 Ry
8+
total all-electron energy = -551.438359 Ry
9+
10+
=------------------------------------------------------------------------------=
11+
JOB DONE.
12+
=------------------------------------------------------------------------------=

‎tests/outputs/test_pw_output.py‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,32 @@ def test_failed(data_regression, to_jsonable, fixture_directory):
7373
)
7474

7575

76+
def test_total_energy_nscf_clobber():
77+
"""Test that `total_energy` falls back to stdout when an nscf XML clobbered the scf one.
78+
79+
QE writes `<etot>0.0</etot>` for `nscf` and `bands` runs (see PW/src/non_scf.f90).
80+
In the common `scf → nscf` workflow at a shared `prefix`, the nscf run overwrites
81+
the scf XML on disk, so the only reliable source of the SCF total energy is the
82+
`! total energy = ...` line in the scf stdout.
83+
"""
84+
from qe_tools import CONSTANTS
85+
86+
pw_directory = Path(__file__).parent / "fixtures" / "pw" / "nscf_etot_clobber"
87+
88+
pw_out = PwOutput.from_dir(pw_directory)
89+
90+
# Sanity check: the fixture really is an nscf XML with the bogus etot.
91+
assert (
92+
pw_out.raw_outputs["xml"]["input"]["control_variables"]["calculation"] == "nscf"
93+
)
94+
assert pw_out.raw_outputs["xml"]["output"]["total_energy"]["etot"] == 0.0
95+
96+
# Despite that, `total_energy` should resolve via the stdout fallback.
97+
assert pw_out.get_output("total_energy") == pytest.approx(
98+
-75.53725762 * CONSTANTS.ry_to_ev
99+
)
100+
101+
76102
def test_insulator_homo(robust_data_regression_check):
77103
"""Test stdout-derived `highest_occupied_level` from an insulator SCF."""
78104

‎tests/outputs/test_pw_output/test_default_xml_211101_.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,7 @@ base_outputs:
9999
raw_outputs:
100100
stdout:
101101
code_version: '7.0'
102+
total_energy: -22.13271119
102103
wall_time_seconds: 1.96
103104
xml:
104105
'@Units': Hartree atomic units

‎tests/outputs/test_pw_output/test_default_xml_220603_.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,7 @@ base_outputs:
115115
raw_outputs:
116116
stdout:
117117
code_version: '7.1'
118+
total_energy: -22.65373597
118119
wall_time_seconds: 2.02
119120
xml:
120121
'@Units': Hartree atomic units

‎tests/outputs/test_pw_output/test_default_xml_230310_.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -167,6 +167,7 @@ raw_outputs:
167167
stdout:
168168
code_version: '7.2'
169169
highest_occupied_level: 9.0922
170+
total_energy: -396.09758618
170171
wall_time_seconds: 100.44
171172
xml:
172173
'@Units': Hartree atomic units

‎tests/outputs/test_pw_output/test_default_xml_240411_.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -151,6 +151,7 @@ base_outputs:
151151
raw_outputs:
152152
stdout:
153153
code_version: 7.3.1
154+
total_energy: -22.66279398
154155
wall_time_seconds: 2.51
155156
xml:
156157
'@Units': Hartree atomic units

‎tests/outputs/test_pw_output/test_default_xml_250521_.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -150,6 +150,7 @@ base_outputs:
150150
raw_outputs:
151151
stdout:
152152
code_version: '7.5'
153+
total_energy: -252.68718165
153154
wall_time_seconds: 2.33
154155
xml:
155156
'@Units': Hartree atomic units

0 commit comments

Comments
 (0)