From e5680cc380c15f8633351a93b30bb50255d214be Mon Sep 17 00:00:00 2001 From: Rahul Mahajan Date: Wed, 28 Feb 2024 13:31:55 -0500 Subject: [PATCH 001/199] Initial commit --- parm/jcb-gdas/LICENSE | 674 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 674 insertions(+) create mode 100644 parm/jcb-gdas/LICENSE diff --git a/parm/jcb-gdas/LICENSE b/parm/jcb-gdas/LICENSE new file mode 100644 index 000000000..f288702d2 --- /dev/null +++ b/parm/jcb-gdas/LICENSE @@ -0,0 +1,674 @@ + GNU GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU General Public License is a free, copyleft license for +software and other kinds of works. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +the GNU General Public License is intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. We, the Free Software Foundation, use the +GNU General Public License for most of our software; it applies also to +any other work released this way by its authors. You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + To protect your rights, we need to prevent others from denying you +these rights or asking you to surrender the rights. Therefore, you have +certain responsibilities if you distribute copies of the software, or if +you modify it: responsibilities to respect the freedom of others. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must pass on to the recipients the same +freedoms that you received. You must make sure that they, too, receive +or can get the source code. And you must show them these terms so they +know their rights. + + Developers that use the GNU GPL protect your rights with two steps: +(1) assert copyright on the software, and (2) offer you this License +giving you legal permission to copy, distribute and/or modify it. + + For the developers' and authors' protection, the GPL clearly explains +that there is no warranty for this free software. For both users' and +authors' sake, the GPL requires that modified versions be marked as +changed, so that their problems will not be attributed erroneously to +authors of previous versions. + + Some devices are designed to deny users access to install or run +modified versions of the software inside them, although the manufacturer +can do so. This is fundamentally incompatible with the aim of +protecting users' freedom to change the software. The systematic +pattern of such abuse occurs in the area of products for individuals to +use, which is precisely where it is most unacceptable. Therefore, we +have designed this version of the GPL to prohibit the practice for those +products. If such problems arise substantially in other domains, we +stand ready to extend this provision to those domains in future versions +of the GPL, as needed to protect the freedom of users. + + Finally, every program is threatened constantly by software patents. +States should not allow patents to restrict development and use of +software on general-purpose computers, but in those that do, we wish to +avoid the special danger that patents applied to a free program could +make it effectively proprietary. To prevent this, the GPL assures that +patents cannot be used to render the program non-free. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Use with the GNU Affero General Public License. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU Affero General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the special requirements of the GNU Affero General Public License, +section 13, concerning interaction through a network will apply to the +combination as such. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . + +Also add information on how to contact you by electronic and paper mail. + + If the program does terminal interaction, make it output a short +notice like this when it starts in an interactive mode: + + Copyright (C) + This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, your program's commands +might be different; for a GUI interface, you would use an "about box". + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU GPL, see +. + + The GNU General Public License does not permit incorporating your program +into proprietary programs. If your program is a subroutine library, you +may consider it more useful to permit linking proprietary applications with +the library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. But first, please read +. From a4d72ce6bc6930e23708b1bd0c2ddcb23d70ea87 Mon Sep 17 00:00:00 2001 From: danholdaway Date: Wed, 28 Feb 2024 15:03:10 -0500 Subject: [PATCH 002/199] Add some yaml files --- parm/jcb-gdas/aero/model/geometry.yaml | 0 parm/jcb-gdas/atmosphere/model/geometry.yaml | 0 parm/jcb-gdas/marine/model/geometry.yaml | 0 parm/jcb-gdas/snow/model/geometry.yaml | 0 4 files changed, 0 insertions(+), 0 deletions(-) create mode 100644 parm/jcb-gdas/aero/model/geometry.yaml create mode 100644 parm/jcb-gdas/atmosphere/model/geometry.yaml create mode 100644 parm/jcb-gdas/marine/model/geometry.yaml create mode 100644 parm/jcb-gdas/snow/model/geometry.yaml diff --git a/parm/jcb-gdas/aero/model/geometry.yaml b/parm/jcb-gdas/aero/model/geometry.yaml new file mode 100644 index 000000000..e69de29bb diff --git a/parm/jcb-gdas/atmosphere/model/geometry.yaml b/parm/jcb-gdas/atmosphere/model/geometry.yaml new file mode 100644 index 000000000..e69de29bb diff --git a/parm/jcb-gdas/marine/model/geometry.yaml b/parm/jcb-gdas/marine/model/geometry.yaml new file mode 100644 index 000000000..e69de29bb diff --git a/parm/jcb-gdas/snow/model/geometry.yaml b/parm/jcb-gdas/snow/model/geometry.yaml new file mode 100644 index 000000000..e69de29bb From 1e9223442286ab843b3e5b5bf627ab74d53f1e3a Mon Sep 17 00:00:00 2001 From: danholdaway Date: Thu, 29 Feb 2024 16:17:58 -0500 Subject: [PATCH 003/199] add some gdas yamls --- .../jcb-gdas/atmosphere/model/background.yaml | 11 + parm/jcb-gdas/atmosphere/model/geometry.yaml | 11 + .../atmosphere/model/model_pseudo.yaml | 14 + .../atmosphere/observations/aircraft.yaml | 330 ++++++++++++++++ .../atmosphere/observations/amsua_n19.yaml | 367 ++++++++++++++++++ 5 files changed, 733 insertions(+) create mode 100644 parm/jcb-gdas/atmosphere/model/background.yaml create mode 100644 parm/jcb-gdas/atmosphere/model/model_pseudo.yaml create mode 100644 parm/jcb-gdas/atmosphere/observations/aircraft.yaml create mode 100644 parm/jcb-gdas/atmosphere/observations/amsua_n19.yaml diff --git a/parm/jcb-gdas/atmosphere/model/background.yaml b/parm/jcb-gdas/atmosphere/model/background.yaml new file mode 100644 index 000000000..936f61182 --- /dev/null +++ b/parm/jcb-gdas/atmosphere/model/background.yaml @@ -0,0 +1,11 @@ +datapath: {{data}}/bkg +filetype: fms restart +datetime: '{{ atm_window_begin_isotime }}' +filename_core: '{{ atm_window_begin_fv3time }}.fv_core.res.nc' +filename_trcr: '{{ atm_window_begin_fv3time }}.fv_tracer.res.nc' +filename_sfcd: '{{ atm_window_begin_fv3time }}.sfc_data.nc' +filename_sfcw: '{{ atm_window_begin_fv3time }}.fv_srf_wnd.res.nc' +filename_cplr: '{{ atm_window_begin_fv3time }}.coupler.res' +state variables: [ua,va,t,delp,sphum,ice_wat,liq_wat,o3mr,phis, + slmsk,sheleg,tsea,vtype,stype,vfrac,stc,smc,snwdph, + u_srf,v_srf,f10m] diff --git a/parm/jcb-gdas/atmosphere/model/geometry.yaml b/parm/jcb-gdas/atmosphere/model/geometry.yaml index e69de29bb..ba5789089 100644 --- a/parm/jcb-gdas/atmosphere/model/geometry.yaml +++ b/parm/jcb-gdas/atmosphere/model/geometry.yaml @@ -0,0 +1,11 @@ +fms initialization: + namelist filename: {{data}}/fv3jedi/fmsmpp.nml + field table filename: {{data}}/fv3jedi/field_table +akbk: {{data}}/fv3jedi/akbk.nc4 +layout: +- {{layout_x}} +- {{layout_y}} +npx: {{npx_ges}} +npy: {{npy_ges}} +npz: {{npz_ges}} +field metadata override: {{data}}/fv3jedi/fv3jedi_fieldmetadata_restart.yaml diff --git a/parm/jcb-gdas/atmosphere/model/model_pseudo.yaml b/parm/jcb-gdas/atmosphere/model/model_pseudo.yaml new file mode 100644 index 000000000..93d3ab7c2 --- /dev/null +++ b/parm/jcb-gdas/atmosphere/model/model_pseudo.yaml @@ -0,0 +1,14 @@ +name: PSEUDO +datapath: {{data}}/bkg +filetype: fms restart +datetime: '{{atm_window_begin_isotime}}' +filename is datetime templated: true +filename_core: '%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc' +filename_trcr: '%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc' +filename_sfcd: '%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc' +filename_sfcw: '%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc' +filename_cplr: '%yyyy%mm%dd.%hh%MM%ss.coupler.res' +tstep: {{bkg_tstep}} +model variables: [ua,va,t,delp,sphum,ice_wat,liq_wat,o3mr,phis, + slmsk,sheleg,tsea,vtype,stype,vfrac,stc,smc,snwdph, + u_srf,v_srf,f10m] diff --git a/parm/jcb-gdas/atmosphere/observations/aircraft.yaml b/parm/jcb-gdas/atmosphere/observations/aircraft.yaml new file mode 100644 index 000000000..5b89dce7a --- /dev/null +++ b/parm/jcb-gdas/atmosphere/observations/aircraft.yaml @@ -0,0 +1,330 @@ +- obs space: + name: Aircraft + obsdatain: + engine: + type: H5File + obsfile: {{data}}/obs/{{oprefix}}aircraft.{{ current_cycle_YMDH }}.nc + obsgrouping: + group variables: ["stationIdentification"] + sort variable: "pressure" + sort order: "descending" + obsdataout: + engine: + type: H5File + obsfile: {{data}}/diags/diag_aircraft_{{ current_cycle_YMDH }}.nc + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward, airTemperature, specificHumidity] + # + obs operator: + name: VertInterp + # + obs filters: + # Observation Range Sanity Check + - filter: Bounds Check + filter variables: + - name: airTemperature + minvalue: 195 + maxvalue: 327 + action: + name: reject + # + - filter: Bounds Check + filter variables: + - name: specificHumidity + minvalue: 1.0E-7 + maxvalue: 0.034999999 + action: + name: reject + # + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130 + maxvalue: 130 + action: + name: reject + # + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130.0 + action: + name: reject + # Reject when pressure is less than 126 mb. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: MetaData/pressure + minvalue: 12600 + action: + name: reject + # + # Reject all obs with PreQC mark already set above 3 + # - filter: PreQC + # maxvalue: 3 + # action: + # name: reject + # + #-------------------------------------------------------------------------------------------------------------------- + # Wind + #-------------------------------------------------------------------------------------------------------------------- + # + # Begin by assigning all ObsError to a constant value. These will get overwritten (as needed) for specific types. + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error parameter: 2.0 # 2.0 m/s + # Assign intial ObsError specific to AIREP/ACARS + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error parameter: 3.6 # 3.6 m/s + where: + - variable: + name: ObsType/windEastward + is_in: 230 + # Assign intial ObsError specific to AMDAR + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error parameter: 3.0 # 3.0 m/s + where: + - variable: + name: ObsType/windEastward + is_in: 231 + # Assign intial ObsError specific to MDCRS + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error parameter: 2.5 # 2.5 m/s + where: + - variable: + name: ObsType/windEastward + is_in: 233 + # + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [100000, 95000, 80000, 65000, 60000, 55000, 50000, 45000, 40000, 35000, 30000, 25000, 20000, 15000, 10000] #Pressure (Pa) + errors: [1.4, 1.5, 1.6, 1.8, 1.9, 2.0, 2.1, 2.3, 2.6, 2.8, 3.0, 3.2, 2.7, 2.4, 2.1] + # Assign the initial ObsError, based on height/pressure for RECON aircraft + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [70000, 65000, 60000, 55000, 50000, 45000, 40000, 35000, 30000, 25000, 20000, 15000, 10000, 7500, 5000] + errors: [2.4, 2.5, 2.6, 2.7, 2.8, 2.95, 3.1, 3.25, 3.4, 3.175, 2.95, 2.725, 2.5, 2.6, 2.7] + where: + - variable: + name: ObsType/windEastward + is_in: 232 + # Reject when difference of wind direction is more than 50 degrees. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/WindDirAngleDiff + options: + minimum_uv: 3.5 + maxvalue: 50.0 + action: + name: reject + defer to post: true + # When multiple obs exist within a single vertical model level, inflate ObsError + # - filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsFunction/ObsErrorFactorConventional + # options: + # test QCflag: PreQC + # inflate variables: [windEastward] + # defer to post: true + # + # - filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsFunction/ObsErrorFactorConventional + # options: + # test QCflag: PreQC + # inflate variables: [windNorthward] + # defer to post: true + # + # Gross error check with (O - B) / ObsError greater than threshold. + - filter: Background Check + filter variables: + - name: windEastward + - name: windNorthward + threshold: 6.0 + absolute threshold: 19.0 + action: + name: reject + # + #-------------------------------------------------------------------------------------------------------------------- + # Temperature + #-------------------------------------------------------------------------------------------------------------------- + # + # Begin by assigning all ObsError to a constant value. These will get overwritten for specific types. + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: assign error + error parameter: 2.0 # 2.0 K + # Assign the initial observation error, based on pressure (for AIREP/ACARS; itype=130) + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [100000, 95000, 90000, 85000, 80000] + errors: [2.5, 2.3, 2.1, 1.9, 1.7] + where: + - variable: + name: ObsType/airTemperature + is_in: 130 + # Assign the initial observation error, based on pressure (for AMDAR and MDCRS; itype=131,133) + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [100000, 95000, 90000, 85000, 80000] + errors: [1.4706, 1.3529, 1.2353, 1.1176, 1.0] + where: + - variable: + name: ObsType/airTemperature + is_in: 131,133 + # Assign the initial observation error, based on pressure (for RECON aircraft; itype=132) + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [100000, 95000, 90000, 85000, 35000, 30000, 25000, 20000, 15000, 10000, 7500, 5000, 4000, 3200, 2000, 1000] + errors: [1.2, 1.1, 0.9, 0.8, 0.8, 0.9, 1.2, 1.2, 1.0, 0.8, 0.8, 0.9, 0.95, 1.0, 1.25, 1.5] + where: + - variable: + name: ObsType/airTemperature + is_in: 132 + # When multiple obs exist within a single vertical model level, inflate ObsError + # - filter: Perform Action + # filter variables: + # - name: airTemperature + # action: + # name: inflate error + # inflation variable: + # name: ObsFunction/ObsErrorFactorConventional + # options: + # test QCflag: PreQC + # inflate variables: [airTemperature] + # defer to post: true + + # Gross error check with (O - B) / ObsError greater than threshold. + - filter: Background Check + filter variables: + - name: airTemperature + threshold: 7.0 + absolute threshold: 8.0 + action: + name: reject + # + #-------------------------------------------------------------------------------------------------------------------- + # Moisture + #-------------------------------------------------------------------------------------------------------------------- + # + # Assign the initial observation error, based on height/pressure ONLY MDCRS + - filter: Perform Action + filter variables: + - name: specificHumidity + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000, 105000, 100000, 95000, 90000, 85000, 80000, 75000, 70000, 65000, 60000, 55000, + 50000, 45000, 40000, 35000, 30000, 25000, 20000, 15000, 10000, 7500, 5000, 4000, 3000] + errors: [.19455, .19062, .18488, .17877, .17342, .16976, .16777, .16696, .16605, .16522, .16637, .17086, + .17791, .18492, .18996, .19294, .19447, .19597, .19748, .19866, .19941, .19979, .19994, .19999, .2] + scale_factor_var: ObsValue/specificHumidity + where: + - variable: + name: ObsType/specificHumidity + is_in: 133 + # When multiple obs exist within a single vertical model level, inflate ObsError + # - filter: Perform Action + # filter variables: + # - name: specificHumidity + # action: + # name: inflate error + # inflation variable: + # name: ObsFunction/ObsErrorFactorConventional + # options: + # test QCflag: PreQC + # inflate variables: [specificHumidity] + # defer to post: true + + # Gross error check with (O - B) / ObsError greater than threshold. + - filter: Background Check + filter variables: + - name: specificHumidity + threshold: 8.0 + action: + name: reject diff --git a/parm/jcb-gdas/atmosphere/observations/amsua_n19.yaml b/parm/jcb-gdas/atmosphere/observations/amsua_n19.yaml new file mode 100644 index 000000000..a4967c196 --- /dev/null +++ b/parm/jcb-gdas/atmosphere/observations/amsua_n19.yaml @@ -0,0 +1,367 @@ +- obs space: + name: amsua_n19 + obsdatain: + engine: + type: H5File + obsfile: {{data}}/obs/{{oprefix}}amsua_n19.{{ current_cycle_YMDH }}.nc + obsdataout: + engine: + type: H5File + obsfile: {{data}}/diags/diag_amsua_n19_{{ current_cycle_YMDH }}.nc + io pool: + max pool size: 1 + simulated variables: [brightnessTemperature] + channels: &amsua_n19_channels 1-15 + obs operator: + name: CRTM + Absorbers: [H2O,O3] + Clouds: [Water, Ice] + Cloud_Fraction: 1.0 + obs options: + Sensor_ID: amsua_n19 + EndianType: little_endian + CoefficientPath: {{data}}/crtm/ + obs bias: + input file: {{data}}/obs/{{gprefix}}amsua_n19.satbias.nc + output file: {{data}}/bc/{{aprefix}}amsua_n19.satbias.nc + variational bc: + predictors: + - name: constant + - name: lapse_rate + order: 2 + tlapse: &amsua_n19_tlapse {{data}}/obs/{{gprefix}}amsua_n19.tlapse.txt + - name: lapse_rate + tlapse: *amsua_n19_tlapse + - name: emissivity + - name: scan_angle + order: 4 + - name: scan_angle + order: 3 + - name: scan_angle + order: 2 + - name: scan_angle + covariance: + minimal required obs number: 20 + variance range: [1.0e-6, 10.0] + step size: 1.0e-4 + largest analysis variance: 10000.0 + prior: + input file: {{data}}/obs/{{gprefix}}amsua_n19.satbias_cov.nc + inflation: + ratio: 1.1 + ratio for small dataset: 2.0 + output file: {{data}}/bc/{{aprefix}}amsua_n19.satbias_cov.nc + obs filters: + - filter: BlackList + filter variables: + - name: brightnessTemperature + channels: *amsua_n19_channels + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelRamp + channels: *amsua_n19_channels + options: + channels: *amsua_n19_channels + xvar: + name: ObsFunction/CLWRetSymmetricMW + options: + clwret_ch238: 1 + clwret_ch314: 2 + clwret_types: [ObsValue, HofX] + x0: [ 0.050, 0.030, 0.030, 0.020, 0.000, + 0.100, 0.000, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.030] + x1: [ 0.600, 0.450, 0.400, 0.450, 1.000, + 1.500, 0.000, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.200] + err0: [ 2.500, 2.200, 2.000, 0.550, 0.300, + 0.230, 0.230, 0.250, 0.250, 0.350, + 0.400, 0.550, 0.800, 3.000, 3.500] + err1: [20.000, 18.000, 12.000, 3.000, 0.500, + 0.300, 0.230, 0.250, 0.250, 0.350, + 0.400, 0.550, 0.800, 3.000, 18.000] + # CLW Retrieval Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: 1-6, 15 + test variables: + - name: ObsFunction/CLWRetMW + options: + clwret_ch238: 1 + clwret_ch314: 2 + clwret_types: [ObsValue] + maxvalue: 999.0 + action: + name: reject + # CLW Retrieval Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: 1-6, 15 + test variables: + - name: ObsFunction/CLWRetMW + options: + clwret_ch238: 1 + clwret_ch314: 2 + clwret_types: [HofX] + maxvalue: 999.0 + action: + name: reject + # Hydrometeor Check (cloud/precipitation affected chanels) + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *amsua_n19_channels + test variables: + - name: ObsFunction/HydrometeorCheckAMSUA + channels: *amsua_n19_channels + options: + channels: *amsua_n19_channels + obserr_clearsky: [ 2.500, 2.200, 2.000, 0.550, 0.300, + 0.230, 0.230, 0.250, 0.250, 0.350, + 0.400, 0.550, 0.800, 3.000, 3.500] + clwret_function: + name: ObsFunction/CLWRetMW + options: + clwret_ch238: 1 + clwret_ch314: 2 + clwret_types: [ObsValue] + obserr_function: + name: ObsFunction/ObsErrorModelRamp + channels: *amsua_n19_channels + options: + channels: *amsua_n19_channels + xvar: + name: ObsFunction/CLWRetSymmetricMW + options: + clwret_ch238: 1 + clwret_ch314: 2 + clwret_types: [ObsValue, HofX] + x0: [ 0.050, 0.030, 0.030, 0.020, 0.000, + 0.100, 0.000, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.030] + x1: [ 0.600, 0.450, 0.400, 0.450, 1.000, + 1.500, 0.000, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.200] + err0: [ 2.500, 2.200, 2.000, 0.550, 0.300, + 0.230, 0.230, 0.250, 0.250, 0.350, + 0.400, 0.550, 0.800, 3.000, 3.500] + err1: [20.000, 18.000, 12.000, 3.000, 0.500, + 0.300, 0.230, 0.250, 0.250, 0.350, + 0.400, 0.550, 0.800, 3.000, 18.000] + maxvalue: 0.0 + action: + name: reject + # Topography check + - filter: BlackList + filter variables: + - name: brightnessTemperature + channels: *amsua_n19_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorTopoRad + channels: *amsua_n19_channels + options: + sensor: amsua_n19 + channels: *amsua_n19_channels + # Transmittnace Top Check + - filter: BlackList + filter variables: + - name: brightnessTemperature + channels: *amsua_n19_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorTransmitTopRad + channels: *amsua_n19_channels + options: + sensor: amsua_n19 + channels: *amsua_n19_channels + # Surface Jacobian check + - filter: BlackList + filter variables: + - name: brightnessTemperature + channels: *amsua_n19_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorSurfJacobianRad + channels: *amsua_n19_channels + options: + sensor: amsua_n19 + channels: *amsua_n19_channels + obserr_demisf: [0.010, 0.020, 0.015, 0.020, 0.200] + obserr_dtempf: [0.500, 2.000, 1.000, 2.000, 4.500] + # Situation dependent Check + - filter: BlackList + filter variables: + - name: brightnessTemperature + channels: *amsua_n19_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorSituDependMW + channels: *amsua_n19_channels + options: + sensor: amsua_n19 + channels: *amsua_n19_channels + clwobs_function: + name: ObsFunction/CLWRetMW + options: + clwret_ch238: 1 + clwret_ch314: 2 + clwret_types: [ObsValue] + clwbkg_function: + name: ObsFunction/CLWRetMW + options: + clwret_ch238: 1 + clwret_ch314: 2 + clwret_types: [HofX] + bias_application: HofX + scatobs_function: + name: ObsFunction/SCATRetMW + options: + scatret_ch238: 1 + scatret_ch314: 2 + scatret_ch890: 15 + scatret_types: [ObsValue] + bias_application: HofX + clwmatchidx_function: + name: ObsFunction/CLWMatchIndexMW + channels: *amsua_n19_channels + options: + channels: *amsua_n19_channels + clwobs_function: + name: ObsFunction/CLWRetMW + options: + clwret_ch238: 1 + clwret_ch314: 2 + clwret_types: [ObsValue] + clwbkg_function: + name: ObsFunction/CLWRetMW + options: + clwret_ch238: 1 + clwret_ch314: 2 + clwret_types: [HofX] + bias_application: HofX + clwret_clearsky: [0.050, 0.030, 0.030, 0.020, 0.000, + 0.100, 0.000, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.030] + obserr_function: + name: ObsFunction/ObsErrorModelRamp + channels: *amsua_n19_channels + options: + channels: *amsua_n19_channels + xvar: + name: ObsFunction/CLWRetSymmetricMW + options: + clwret_ch238: 1 + clwret_ch314: 2 + clwret_types: [ObsValue, HofX] + x0: [ 0.050, 0.030, 0.030, 0.020, 0.000, + 0.100, 0.000, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.030] + x1: [ 0.600, 0.450, 0.400, 0.450, 1.000, + 1.500, 0.000, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.200] + err0: [ 2.500, 2.200, 2.000, 0.550, 0.300, + 0.230, 0.230, 0.250, 0.250, 0.350, + 0.400, 0.550, 0.800, 3.000, 3.500] + err1: [20.000, 18.000, 12.000, 3.000, 0.500, + 0.300, 0.230, 0.250, 0.250, 0.350, + 0.400, 0.550, 0.800, 3.000, 18.000] + obserr_clearsky: [2.500, 2.200, 2.000, 0.550, 0.300, + 0.230, 0.230, 0.250, 0.250, 0.350, + 0.400, 0.550, 0.800, 3.000, 3.500] + # Gross check + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: *amsua_n19_channels + function absolute threshold: + - name: ObsFunction/ObsErrorBoundMW + channels: *amsua_n19_channels + options: + sensor: amsua_n19 + channels: *amsua_n19_channels + obserr_bound_latitude: + name: ObsFunction/ObsErrorFactorLatRad + options: + latitude_parameters: [25.0, 0.25, 0.04, 3.0] + obserr_bound_transmittop: + name: ObsFunction/ObsErrorFactorTransmitTopRad + channels: *amsua_n19_channels + options: + channels: *amsua_n19_channels + obserr_bound_topo: + name: ObsFunction/ObsErrorFactorTopoRad + channels: *amsua_n19_channels + options: + channels: *amsua_n19_channels + sensor: amsua_n19 + obserr_function: + name: ObsFunction/ObsErrorModelRamp + channels: *amsua_n19_channels + options: + channels: *amsua_n19_channels + xvar: + name: ObsFunction/CLWRetSymmetricMW + options: + clwret_ch238: 1 + clwret_ch314: 2 + clwret_types: [ObsValue, HofX] + bias_application: HofX + x0: [ 0.050, 0.030, 0.030, 0.020, 0.000, + 0.100, 0.000, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.030] + x1: [ 0.600, 0.450, 0.400, 0.450, 1.000, + 1.500, 0.000, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.200] + err0: [ 2.500, 2.200, 2.000, 0.550, 0.300, + 0.230, 0.230, 0.250, 0.250, 0.350, + 0.400, 0.550, 0.800, 3.000, 3.500] + err1: [20.000, 18.000, 12.000, 3.000, 0.500, + 0.300, 0.230, 0.250, 0.250, 0.350, + 0.400, 0.550, 0.800, 3.000, 18.000] + obserr_bound_max: [4.5, 4.5, 4.5, 2.5, 2.0, + 2.0, 2.0, 2.0, 2.0, 2.0, + 2.5, 3.5, 4.5, 4.5, 4.5] + action: + name: reject + # Inter-channel check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *amsua_n19_channels + test variables: + - name: ObsFunction/InterChannelConsistencyCheck + channels: *amsua_n19_channels + options: + channels: *amsua_n19_channels + sensor: amsua_n19 + use_flag: [ 1, 1, 1, 1, 1, + 1, -1, -1, 1, 1, + 1, 1, 1, -1, 1 ] + maxvalue: 1.0e-12 + action: + name: reject + # Useflag check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *amsua_n19_channels + test variables: + - name: ObsFunction/ChannelUseflagCheckRad + channels: *amsua_n19_channels + options: + sensor: amsua_n19 + channels: *amsua_n19_channels + use_flag: [ 1, 1, 1, 1, 1, + 1, -1, -1, 1, 1, + 1, 1, 1, -1, 1 ] + minvalue: 1.0e-12 + action: + name: reject From 26ccfca86cfdd967d8cc93d5b78f8e9dc657630a Mon Sep 17 00:00:00 2001 From: danholdaway Date: Fri, 8 Mar 2024 11:48:04 -0500 Subject: [PATCH 004/199] add berr files from gdasapp --- .../atmosphere/model/3dvar_outer_loop_1.yaml | 0 .../atmosphere/model/3dvar_outer_loop_2.yaml | 0 .../background_error_hybrid_bump_bump.yaml | 127 +++++++++++ .../background_error_hybrid_gsibec_bump.yaml | 205 ++++++++++++++++++ .../model/background_error_static_bump.yaml | 89 ++++++++ .../model/background_error_static_gsibec.yaml | 26 +++ .../background_error_static_identity.yaml | 2 + .../atmosphere/model/final_increment.yaml | 0 ...geometry.yaml => geometry_background.yaml} | 0 9 files changed, 449 insertions(+) create mode 100644 parm/jcb-gdas/atmosphere/model/3dvar_outer_loop_1.yaml create mode 100644 parm/jcb-gdas/atmosphere/model/3dvar_outer_loop_2.yaml create mode 100644 parm/jcb-gdas/atmosphere/model/background_error_hybrid_bump_bump.yaml create mode 100644 parm/jcb-gdas/atmosphere/model/background_error_hybrid_gsibec_bump.yaml create mode 100644 parm/jcb-gdas/atmosphere/model/background_error_static_bump.yaml create mode 100644 parm/jcb-gdas/atmosphere/model/background_error_static_gsibec.yaml create mode 100644 parm/jcb-gdas/atmosphere/model/background_error_static_identity.yaml create mode 100644 parm/jcb-gdas/atmosphere/model/final_increment.yaml rename parm/jcb-gdas/atmosphere/model/{geometry.yaml => geometry_background.yaml} (100%) diff --git a/parm/jcb-gdas/atmosphere/model/3dvar_outer_loop_1.yaml b/parm/jcb-gdas/atmosphere/model/3dvar_outer_loop_1.yaml new file mode 100644 index 000000000..e69de29bb diff --git a/parm/jcb-gdas/atmosphere/model/3dvar_outer_loop_2.yaml b/parm/jcb-gdas/atmosphere/model/3dvar_outer_loop_2.yaml new file mode 100644 index 000000000..e69de29bb diff --git a/parm/jcb-gdas/atmosphere/model/background_error_hybrid_bump_bump.yaml b/parm/jcb-gdas/atmosphere/model/background_error_hybrid_bump_bump.yaml new file mode 100644 index 000000000..11398dccd --- /dev/null +++ b/parm/jcb-gdas/atmosphere/model/background_error_hybrid_bump_bump.yaml @@ -0,0 +1,127 @@ +covariance model: hybrid +components: +- covariance: + covariance model: SABER + saber blocks: + - saber block name: BUMP_NICAS + saber central block: true + input variables: &control_vars [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] + output variables: *control_vars + active variables: &active_vars [psi,chi,t,ps,rh,liq_wat,o3mr] + bump: + io: + data directory: &staticb_dir $(DATA)/berror + drivers: + multivariate strategy: specific_univariate + read local nicas: true + nicas: + minimum level: + - variables: + - liq_wat + value: 76 + grids: + - model: + variables: + - stream_function + - velocity_potential + - air_temperature + - relative_humidity + - cloud_liquid_water + - ozone_mass_mixing_ratio + io: + files prefix: nicas/nicas_3D_gfs + - model: + variables: + - surface_pressure + io: + files prefix: nicas/nicas_2D_gfs + - saber block name: StdDev + input variables: *control_vars + output variables: *control_vars + active variables: *active_vars + file: + filetype: fms restart + datetime: 2021-12-22T00:00:00Z + set datetime on read: true + psinfile: true + datapath: '*staticb_dir' + prefix: stddev/20211222.000000 + filename_core: stddev.fv_core.res.nc + filename_trcr: stddev.fv_tracer.res.nc + filename_sfcd: stddev.sfc_data.nc + filename_sfcw: stddev.fv_srf_wnd.res.nc + filename_cplr: stddev.coupler.res + - saber block name: BUMP_VerticalBalance + input variables: *control_vars + output variables: *control_vars + active variables: *active_vars + bump: + general: + universe length-scale: 2000.0e3 + io: + data directory: *staticb_dir + files prefix: vbal/vbal_gfs + overriding sampling file: vbal/vbal_2021122200_gfs_sampling + drivers: + read local sampling: true + read vertical balance: true + vertical balance: + vbal: + - balanced variable: var2 + unbalanced variable: var1 + - balanced variable: var3 + unbalanced variable: var1 + - balanced variable: var4 + unbalanced variable: var1 + - saber block name: BUMP_PsiChiToUV + input variables: *control_vars + output variables: &3dvars_with_rh [ua,va,t,ps,rh,ice_wat,liq_wat,o3mr] + active variables: [psi,chi,ua,va] + bump: + general: + universe length-scale: 2000.0e3 + io: + data directory: *staticb_dir + files prefix: psichitouv/psichitouv_gfs + drivers: + read local psichitouv: true + linear variable change: + linear variable change name: Control2Analysis + input variables: *3dvars_with_rh + output variables: &3dvars [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] + weight: + value: 0.1 +- covariance: + covariance model: ensemble + members from template: + template: + datetime: '{{ current_cycle | to_isotime }}' + filetype: fms restart + state variables: *3dvars + datapath: ens/mem%mem%/ + filename_core: '{{ current_cycle | to_fv3time }}.fv_core.res.nc' + filename_trcr: '{{ current_cycle | to_fv3time }}.fv_tracer.res.nc' + filename_sfcd: '{{ current_cycle | to_fv3time }}.sfc_data.nc' + filename_sfcw: '{{ current_cycle | to_fv3time }}.fv_srf_wnd.res.nc' + filename_cplr: '{{ current_cycle | to_fv3time }}.coupler.res' + pattern: '%mem%' + nmembers: $(NMEM_ENS) + zero padding: 3 +# localization: +# localization method: SABER +# saber block: +# - saber block name: BUMP_NICAS +# input variables: *control_vars +# output variables: *control_vars +# active variables: *active_vars +# bump: +# datadir: *staticb_dir +# verbosity: main +# strategy: specific_univariate +# method: loc +# load_nicas_local: true +# grids: +# - prefix: nicas/nicas_3D_gfs +# variables: [stream_function,velocity_potential,air_temperature,relative_humidity,cloud_liquid_water,ozone_mass_mixing_ratio] + weight: + value: 0.9 diff --git a/parm/jcb-gdas/atmosphere/model/background_error_hybrid_gsibec_bump.yaml b/parm/jcb-gdas/atmosphere/model/background_error_hybrid_gsibec_bump.yaml new file mode 100644 index 000000000..356b22cec --- /dev/null +++ b/parm/jcb-gdas/atmosphere/model/background_error_hybrid_gsibec_bump.yaml @@ -0,0 +1,205 @@ +covariance model: hybrid +components: +- covariance: + covariance model: SABER + saber central block: + saber block name: gsi covariance + read: + gsi akbk: &akbk $(DATA)/fv3jedi/akbk.nc4 + gsi error covariance file: &gsiberr $(DATA)/berror/gsi-coeffs-gfs-global.nc4 +# gsi error covariance file: &gsiberr $(DATA)/berror/global_berror.f77 + gsi berror namelist file: &gsibnml $(DATA)/berror/gfs_gsi_global.nml + processor layout x direction: &layout_gsib_x $(layout_gsib_x) + processor layout y direction: &layout_gsib_y $(layout_gsib_y) + debugging mode: false + saber outer blocks: + - saber block name: gsi interpolation to model grid + gsi akbk: *akbk + gsi error covariance file: *gsiberr + gsi berror namelist file: *gsibnml + processor layout x direction: *layout_gsib_x + processor layout y direction: *layout_gsib_y + debugging mode: false + linear variable change: + linear variable change name: Control2Analysis + input variables: &control_vars [eastward_wind,northward_wind,air_temperature,surface_pressure, + specific_humidity,cloud_liquid_ice,cloud_liquid_water, + mole_fraction_of_ozone_in_air] + output variables: &3dvars_anal [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] + weight: + value: 0.125 +- covariance: + covariance model: ensemble + members from template: + template: + datetime: '{{ current_cycle | to_isotime }}' + filetype: fms restart + state variables: *3dvars_anal + datapath: ens/mem%mem% + filename_core: '{{ current_cycle | to_fv3time }}.fv_core.res.nc' + filename_trcr: '{{ current_cycle | to_fv3time }}.fv_tracer.res.nc' + filename_sfcd: '{{ current_cycle | to_fv3time }}.sfc_data.nc' + filename_sfcw: '{{ current_cycle | to_fv3time }}.fv_srf_wnd.res.nc' + filename_cplr: '{{ current_cycle | to_fv3time }}.coupler.res' + pattern: '%mem%' + nmembers: $(NMEM_ENS) + zero padding: 3 + localization: + localization method: SABER + saber central block: + saber block name: BUMP_NICAS + active variables: &3dvars_anal_long [eastward_wind,northward_wind,air_temperature,surface_pressure, + specific_humidity,cloud_liquid_ice,cloud_liquid_water, + ozone_mass_mixing_ratio ] + read: + general: + universe length-scale: 2500.0e3 + drivers: + multivariate strategy: duplicated + compute nicas: true + model: + level for 2d variables: last + nicas: + resolution: 6 + explicit length-scales: true + horizontal length-scale: + - groups: + - common + profile: + - 1300000.0 + - 1300000.0 + - 1300000.0 + - 1300000.0 + - 1300000.0 + - 1300000.0 + - 1300000.0 + - 1300000.0 + - 1300000.0 + - 1300000.0 + - 1300000.0 + - 1250000.0 + - 1250000.0 + - 1200000.0 + - 1200000.0 + - 1150000.0 + - 1150000.0 + - 1100000.0 + - 1100000.0 + - 1000000.0 + - 1000000.0 + - 1000000.0 + - 1000000.0 + - 1000000.0 + - 1000000.0 + - 1000000.0 + - 1000000.0 + - 1000000.0 + - 1000000.0 + - 1000000.0 + - 1000000.0 + - 1000000.0 + - 1000000.0 + - 1000000.0 + - 1000000.0 + - 1000000.0 + - 1000000.0 + - 1000000.0 + - 1000000.0 + - 950000.0 + - 950000.0 + - 900000.0 + - 900000.0 + - 850000.0 + - 850000.0 + - 800000.0 + - 800000.0 + - 750000.0 + - 750000.0 + - 665000.0 + - 665000.0 + - 585000.0 + - 585000.0 + - 510000.0 + - 510000.0 + - 440000.0 + - 440000.0 + - 390000.0 + - 390000.0 + - 380000.0 + - 380000.0 + - 370000.0 + - 370000.0 + - 360000.0 + - 360000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + - 350000.0 + vertical length-scale: + - groups: + - common + value: 0.3 + linear variable change: + linear variable change name: Control2Analysis + input variables: *3dvars_anal_long + output variables: *3dvars_anal + weight: + value: 0.875 diff --git a/parm/jcb-gdas/atmosphere/model/background_error_static_bump.yaml b/parm/jcb-gdas/atmosphere/model/background_error_static_bump.yaml new file mode 100644 index 000000000..8cabf7060 --- /dev/null +++ b/parm/jcb-gdas/atmosphere/model/background_error_static_bump.yaml @@ -0,0 +1,89 @@ +covariance model: SABER +full inverse: true +saber blocks: +- saber block name: BUMP_NICAS + saber central block: true + input variables: &control_vars [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] + output variables: *control_vars + active variables: &active_vars [psi,chi,t,ps,rh,liq_wat,o3mr] + bump: + io: + data directory: &staticb_dir $(DATA)/berror + drivers: + multivariate strategy: specific_univariate + read local nicas: true + nicas: + minimum level: + - variables: + - liq_wat + value: 76 + grids: + - model: + variables: + - stream_function + - velocity_potential + - air_temperature + - relative_humidity + - cloud_liquid_water + - ozone_mass_mixing_ratio + io: + files prefix: nicas/nicas_3D_gfs + - model: + variables: + - surface_pressure + io: + files prefix: nicas/nicas_2D_gfs +- saber block name: StdDev + input variables: *control_vars + output variables: *control_vars + active variables: *active_vars + file: + filetype: fms restart + datetime: 2021-12-22T00:00:00Z + set datetime on read: true + psinfile: true + datapath: *staticb_dir + prefix: stddev/20211222.000000 + filename_core: stddev.fv_core.res.nc + filename_trcr: stddev.fv_tracer.res.nc + filename_sfcd: stddev.sfc_data.nc + filename_sfcw: stddev.fv_srf_wnd.res.nc + filename_cplr: stddev.coupler.res +- saber block name: BUMP_VerticalBalance + input variables: *control_vars + output variables: *control_vars + active variables: *active_vars + bump: + general: + universe length-scale: 2000.0e3 + io: + data directory: *staticb_dir + files prefix: vbal/vbal_gfs + overriding sampling file: vbal/vbal_2021122200_gfs_sampling + drivers: + read local sampling: true + read vertical balance: true + vertical balance: + vbal: + - balanced variable: var2 + unbalanced variable: var1 + - balanced variable: var3 + unbalanced variable: var1 + - balanced variable: var4 + unbalanced variable: var1 +- saber block name: BUMP_PsiChiToUV + input variables: *control_vars + output variables: &3dvars_with_rh [ua,va,t,ps,rh,ice_wat,liq_wat,o3mr] + active variables: [psi,chi,ua,va] + bump: + general: + universe length-scale: 2000.0e3 + io: + data directory: '*staticb_dir' + files prefix: psichitouv/psichitouv_gfs + drivers: + read local psichitouv: true +linear variable change: + linear variable change name: Control2Analysis + input variables: *3dvars_with_rh + output variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] diff --git a/parm/jcb-gdas/atmosphere/model/background_error_static_gsibec.yaml b/parm/jcb-gdas/atmosphere/model/background_error_static_gsibec.yaml new file mode 100644 index 000000000..ba218842e --- /dev/null +++ b/parm/jcb-gdas/atmosphere/model/background_error_static_gsibec.yaml @@ -0,0 +1,26 @@ +covariance model: SABER +full inverse: true +saber central block: + saber block name: gsi covariance + read: + gsi akbk: &akbk $(DATA)/fv3jedi/akbk.nc4 + gsi error covariance file: &gsiberr $(DATA)/berror/gsi-coeffs-gfs-global.nc4 +# gsi error covariance file: &gsiberr $(DATA)/berror/global_berror.f77 + gsi berror namelist file: &gsibnml $(DATA)/berror/gfs_gsi_global.nml + processor layout x direction: &layout_gsib_x $(layout_gsib_x) + processor layout y direction: &layout_gsib_y $(layout_gsib_y) + debugging mode: false +saber outer blocks: +- saber block name: gsi interpolation to model grid + gsi akbk: *akbk + gsi error covariance file: *gsiberr + gsi berror namelist file: *gsibnml + processor layout x direction: *layout_gsib_x + processor layout y direction: *layout_gsib_y + debugging mode: false +linear variable change: + linear variable change name: Control2Analysis + input variables: &control_vars [eastward_wind,northward_wind,air_temperature,surface_pressure, + specific_humidity,cloud_liquid_ice,cloud_liquid_water, + mole_fraction_of_ozone_in_air] + output variables: &3dvars_anal [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] diff --git a/parm/jcb-gdas/atmosphere/model/background_error_static_identity.yaml b/parm/jcb-gdas/atmosphere/model/background_error_static_identity.yaml new file mode 100644 index 000000000..62fc984c6 --- /dev/null +++ b/parm/jcb-gdas/atmosphere/model/background_error_static_identity.yaml @@ -0,0 +1,2 @@ +covariance model: FV3JEDI-ID +date: '{{ATM_WINDOW_BEGIN}}' diff --git a/parm/jcb-gdas/atmosphere/model/final_increment.yaml b/parm/jcb-gdas/atmosphere/model/final_increment.yaml new file mode 100644 index 000000000..e69de29bb diff --git a/parm/jcb-gdas/atmosphere/model/geometry.yaml b/parm/jcb-gdas/atmosphere/model/geometry_background.yaml similarity index 100% rename from parm/jcb-gdas/atmosphere/model/geometry.yaml rename to parm/jcb-gdas/atmosphere/model/geometry_background.yaml From 72cd806f8ef137e947edf5281807196cdd1de041 Mon Sep 17 00:00:00 2001 From: danholdaway Date: Fri, 8 Mar 2024 11:51:28 -0500 Subject: [PATCH 005/199] remove filters --- .../model/background_error_hybrid_bump_bump.yaml | 14 +++++++------- .../model/background_error_hybrid_gsibec_bump.yaml | 12 ++++++------ 2 files changed, 13 insertions(+), 13 deletions(-) diff --git a/parm/jcb-gdas/atmosphere/model/background_error_hybrid_bump_bump.yaml b/parm/jcb-gdas/atmosphere/model/background_error_hybrid_bump_bump.yaml index 11398dccd..8c3d21475 100644 --- a/parm/jcb-gdas/atmosphere/model/background_error_hybrid_bump_bump.yaml +++ b/parm/jcb-gdas/atmosphere/model/background_error_hybrid_bump_bump.yaml @@ -95,17 +95,17 @@ components: covariance model: ensemble members from template: template: - datetime: '{{ current_cycle | to_isotime }}' + datetime: '{{ current_cycle_isotime }}' filetype: fms restart state variables: *3dvars datapath: ens/mem%mem%/ - filename_core: '{{ current_cycle | to_fv3time }}.fv_core.res.nc' - filename_trcr: '{{ current_cycle | to_fv3time }}.fv_tracer.res.nc' - filename_sfcd: '{{ current_cycle | to_fv3time }}.sfc_data.nc' - filename_sfcw: '{{ current_cycle | to_fv3time }}.fv_srf_wnd.res.nc' - filename_cplr: '{{ current_cycle | to_fv3time }}.coupler.res' + filename_core: '{{ current_cycle_fv3time }}.fv_core.res.nc' + filename_trcr: '{{ current_cycle_fv3time }}.fv_tracer.res.nc' + filename_sfcd: '{{ current_cycle_fv3time }}.sfc_data.nc' + filename_sfcw: '{{ current_cycle_fv3time }}.fv_srf_wnd.res.nc' + filename_cplr: '{{ current_cycle_fv3time }}.coupler.res' pattern: '%mem%' - nmembers: $(NMEM_ENS) + nmembers: {{number_ensemble_members}} zero padding: 3 # localization: # localization method: SABER diff --git a/parm/jcb-gdas/atmosphere/model/background_error_hybrid_gsibec_bump.yaml b/parm/jcb-gdas/atmosphere/model/background_error_hybrid_gsibec_bump.yaml index 356b22cec..3c6fde5a6 100644 --- a/parm/jcb-gdas/atmosphere/model/background_error_hybrid_gsibec_bump.yaml +++ b/parm/jcb-gdas/atmosphere/model/background_error_hybrid_gsibec_bump.yaml @@ -32,15 +32,15 @@ components: covariance model: ensemble members from template: template: - datetime: '{{ current_cycle | to_isotime }}' + datetime: '{{ current_cycle_isotime }}' filetype: fms restart state variables: *3dvars_anal datapath: ens/mem%mem% - filename_core: '{{ current_cycle | to_fv3time }}.fv_core.res.nc' - filename_trcr: '{{ current_cycle | to_fv3time }}.fv_tracer.res.nc' - filename_sfcd: '{{ current_cycle | to_fv3time }}.sfc_data.nc' - filename_sfcw: '{{ current_cycle | to_fv3time }}.fv_srf_wnd.res.nc' - filename_cplr: '{{ current_cycle | to_fv3time }}.coupler.res' + filename_core: '{{ current_cycle_fv3time }}.fv_core.res.nc' + filename_trcr: '{{ current_cycle_fv3time }}.fv_tracer.res.nc' + filename_sfcd: '{{ current_cycle_fv3time }}.sfc_data.nc' + filename_sfcw: '{{ current_cycle_fv3time }}.fv_srf_wnd.res.nc' + filename_cplr: '{{ current_cycle_fv3time }}.coupler.res' pattern: '%mem%' nmembers: $(NMEM_ENS) zero padding: 3 From f50c65387551749ac3f30cfd31edd49404253084 Mon Sep 17 00:00:00 2001 From: danholdaway Date: Fri, 8 Mar 2024 12:36:50 -0500 Subject: [PATCH 006/199] 3dvar looks to be working --- .../atmosphere/model/3dvar_outer_loop_1.yaml | 17 +++++++++++++++++ .../atmosphere/model/3dvar_outer_loop_2.yaml | 17 +++++++++++++++++ .../background_error_hybrid_bump_bump.yaml | 2 +- .../background_error_hybrid_gsibec_bump.yaml | 14 +++++++------- .../model/background_error_static_bump.yaml | 2 +- .../model/background_error_static_gsibec.yaml | 12 ++++++------ .../atmosphere/model/final_increment.yaml | 17 +++++++++++++++++ 7 files changed, 66 insertions(+), 15 deletions(-) diff --git a/parm/jcb-gdas/atmosphere/model/3dvar_outer_loop_1.yaml b/parm/jcb-gdas/atmosphere/model/3dvar_outer_loop_1.yaml index e69de29bb..566360652 100644 --- a/parm/jcb-gdas/atmosphere/model/3dvar_outer_loop_1.yaml +++ b/parm/jcb-gdas/atmosphere/model/3dvar_outer_loop_1.yaml @@ -0,0 +1,17 @@ +- ninner: 2 + gradient norm reduction: 1e-10 + test: on + geometry: + fms initialization: + namelist filename: ./fv3jedi/fmsmpp.nml + field table filename: ./fv3jedi/field_table + akbk: ./fv3jedi/akbk.nc4 + layout: + - {{layout_x}} + - {{layout_y}} + npx: {{npx_anl}} + npy: {{npy_anl}} + npz: {{npz_anl}} + field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_restart.yaml + diagnostics: + departures: bkgmob diff --git a/parm/jcb-gdas/atmosphere/model/3dvar_outer_loop_2.yaml b/parm/jcb-gdas/atmosphere/model/3dvar_outer_loop_2.yaml index e69de29bb..8044f7f9c 100644 --- a/parm/jcb-gdas/atmosphere/model/3dvar_outer_loop_2.yaml +++ b/parm/jcb-gdas/atmosphere/model/3dvar_outer_loop_2.yaml @@ -0,0 +1,17 @@ +- ninner: 4 + gradient norm reduction: 1e-10 + test: on + geometry: + fms initialization: + namelist filename: ./fv3jedi/fmsmpp.nml + field table filename: ./fv3jedi/field_table + akbk: ./fv3jedi/akbk.nc4 + layout: + - {{layout_x}} + - {{layout_y}} + npx: {{npx_anl}} + npy: {{npy_anl}} + npz: {{npz_anl}} + field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_restart.yaml + diagnostics: + departures: bkgmob1 diff --git a/parm/jcb-gdas/atmosphere/model/background_error_hybrid_bump_bump.yaml b/parm/jcb-gdas/atmosphere/model/background_error_hybrid_bump_bump.yaml index 8c3d21475..0c65415af 100644 --- a/parm/jcb-gdas/atmosphere/model/background_error_hybrid_bump_bump.yaml +++ b/parm/jcb-gdas/atmosphere/model/background_error_hybrid_bump_bump.yaml @@ -10,7 +10,7 @@ components: active variables: &active_vars [psi,chi,t,ps,rh,liq_wat,o3mr] bump: io: - data directory: &staticb_dir $(DATA)/berror + data directory: &staticb_dir {{data}}/berror drivers: multivariate strategy: specific_univariate read local nicas: true diff --git a/parm/jcb-gdas/atmosphere/model/background_error_hybrid_gsibec_bump.yaml b/parm/jcb-gdas/atmosphere/model/background_error_hybrid_gsibec_bump.yaml index 3c6fde5a6..926098a46 100644 --- a/parm/jcb-gdas/atmosphere/model/background_error_hybrid_gsibec_bump.yaml +++ b/parm/jcb-gdas/atmosphere/model/background_error_hybrid_gsibec_bump.yaml @@ -5,12 +5,12 @@ components: saber central block: saber block name: gsi covariance read: - gsi akbk: &akbk $(DATA)/fv3jedi/akbk.nc4 - gsi error covariance file: &gsiberr $(DATA)/berror/gsi-coeffs-gfs-global.nc4 -# gsi error covariance file: &gsiberr $(DATA)/berror/global_berror.f77 - gsi berror namelist file: &gsibnml $(DATA)/berror/gfs_gsi_global.nml - processor layout x direction: &layout_gsib_x $(layout_gsib_x) - processor layout y direction: &layout_gsib_y $(layout_gsib_y) + gsi akbk: &akbk {{data}}/fv3jedi/akbk.nc4 + gsi error covariance file: &gsiberr {{data}}/berror/gsi-coeffs-gfs-global.nc4 +# gsi error covariance file: &gsiberr {{data}}/berror/global_berror.f77 + gsi berror namelist file: &gsibnml {{data}}/berror/gfs_gsi_global.nml + processor layout x direction: &layout_gsib_x {{layout_gsib_x}} + processor layout y direction: &layout_gsib_y {{layout_gsib_y}} debugging mode: false saber outer blocks: - saber block name: gsi interpolation to model grid @@ -42,7 +42,7 @@ components: filename_sfcw: '{{ current_cycle_fv3time }}.fv_srf_wnd.res.nc' filename_cplr: '{{ current_cycle_fv3time }}.coupler.res' pattern: '%mem%' - nmembers: $(NMEM_ENS) + nmembers: {{number_ensemble_members}} zero padding: 3 localization: localization method: SABER diff --git a/parm/jcb-gdas/atmosphere/model/background_error_static_bump.yaml b/parm/jcb-gdas/atmosphere/model/background_error_static_bump.yaml index 8cabf7060..9dc16f7cf 100644 --- a/parm/jcb-gdas/atmosphere/model/background_error_static_bump.yaml +++ b/parm/jcb-gdas/atmosphere/model/background_error_static_bump.yaml @@ -8,7 +8,7 @@ saber blocks: active variables: &active_vars [psi,chi,t,ps,rh,liq_wat,o3mr] bump: io: - data directory: &staticb_dir $(DATA)/berror + data directory: &staticb_dir {{data}}/berror drivers: multivariate strategy: specific_univariate read local nicas: true diff --git a/parm/jcb-gdas/atmosphere/model/background_error_static_gsibec.yaml b/parm/jcb-gdas/atmosphere/model/background_error_static_gsibec.yaml index ba218842e..2d8d01a8d 100644 --- a/parm/jcb-gdas/atmosphere/model/background_error_static_gsibec.yaml +++ b/parm/jcb-gdas/atmosphere/model/background_error_static_gsibec.yaml @@ -3,12 +3,12 @@ full inverse: true saber central block: saber block name: gsi covariance read: - gsi akbk: &akbk $(DATA)/fv3jedi/akbk.nc4 - gsi error covariance file: &gsiberr $(DATA)/berror/gsi-coeffs-gfs-global.nc4 -# gsi error covariance file: &gsiberr $(DATA)/berror/global_berror.f77 - gsi berror namelist file: &gsibnml $(DATA)/berror/gfs_gsi_global.nml - processor layout x direction: &layout_gsib_x $(layout_gsib_x) - processor layout y direction: &layout_gsib_y $(layout_gsib_y) + gsi akbk: &akbk {{data}}/fv3jedi/akbk.nc4 + gsi error covariance file: &gsiberr {{data}}/berror/gsi-coeffs-gfs-global.nc4 +# gsi error covariance file: &gsiberr {{data}}/berror/global_berror.f77 + gsi berror namelist file: &gsibnml {{data}}/berror/gfs_gsi_global.nml + processor layout x direction: &layout_gsib_x {{layout_gsib_x}} + processor layout y direction: &layout_gsib_y {{layout_gsib_y}} debugging mode: false saber outer blocks: - saber block name: gsi interpolation to model grid diff --git a/parm/jcb-gdas/atmosphere/model/final_increment.yaml b/parm/jcb-gdas/atmosphere/model/final_increment.yaml index e69de29bb..901fed375 100644 --- a/parm/jcb-gdas/atmosphere/model/final_increment.yaml +++ b/parm/jcb-gdas/atmosphere/model/final_increment.yaml @@ -0,0 +1,17 @@ +output: + state component: + filetype: auxgrid + gridtype: gaussian + filename: ./anl/atminc. +geometry: + fms initialization: + namelist filename: ./fv3jedi/fmsmpp.nml + field table filename: ./fv3jedi/field_table + akbk: ./fv3jedi/akbk.nc4 + layout: + - {{layout_x}} + - {{layout_y}} + npx: {{npx_anl}} + npy: {{npy_anl}} + npz: {{npz_anl}} + field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_restart.yaml From 811a74e8fac504db0c6b67334ee8dd92a36c2785 Mon Sep 17 00:00:00 2001 From: danholdaway Date: Tue, 12 Mar 2024 15:41:08 -0400 Subject: [PATCH 007/199] add building blocks for ensemble da --- .../atmosphere/model/background_ensemble.yaml | 17 +++++++++++++++++ .../model/output_ensemble_increments.yaml | 3 +++ .../atmosphere/model/posterior_output.yaml | 3 +++ .../atmosphere/observations/aircraft.yaml | 8 ++++++++ .../atmosphere/observations/amsua_n19.yaml | 8 ++++++++ 5 files changed, 39 insertions(+) create mode 100644 parm/jcb-gdas/atmosphere/model/background_ensemble.yaml create mode 100644 parm/jcb-gdas/atmosphere/model/output_ensemble_increments.yaml create mode 100644 parm/jcb-gdas/atmosphere/model/posterior_output.yaml diff --git a/parm/jcb-gdas/atmosphere/model/background_ensemble.yaml b/parm/jcb-gdas/atmosphere/model/background_ensemble.yaml new file mode 100644 index 000000000..87a26ed54 --- /dev/null +++ b/parm/jcb-gdas/atmosphere/model/background_ensemble.yaml @@ -0,0 +1,17 @@ +datetime: '{{ current_cycle_isotime }}' +members from template: + template: + datetime: '{{ current_cycle_isotime }}' + filetype: fms restart + state variables: [ua,va,t,DZ,delp,ps,sphum,ice_wat,liq_wat,o3mr,phis, + slmsk,sheleg,tsea,vtype,stype,vfrac,stc,smc,snwdph, + u_srf,v_srf,f10m] + datapath: bkg/mem%mem% + filename_core: '{{ current_cycle_fv3time }}.fv_core.res.nc' + filename_trcr: '{{ current_cycle_fv3time }}.fv_tracer.res.nc' + filename_sfcd: '{{ current_cycle_fv3time }}.sfc_data.nc' + filename_sfcw: '{{ current_cycle_fv3time }}.fv_srf_wnd.res.nc' + filename_cplr: '{{ current_cycle_fv3time }}.coupler.res' + pattern: '%mem%' + nmembers: {{ number_ensemble_members }} + zero padding: 3 diff --git a/parm/jcb-gdas/atmosphere/model/output_ensemble_increments.yaml b/parm/jcb-gdas/atmosphere/model/output_ensemble_increments.yaml new file mode 100644 index 000000000..7f9952872 --- /dev/null +++ b/parm/jcb-gdas/atmosphere/model/output_ensemble_increments.yaml @@ -0,0 +1,3 @@ +filetype: auxgrid +gridtype: gaussian +filename: ./anl/mem%{member}%/atminc. diff --git a/parm/jcb-gdas/atmosphere/model/posterior_output.yaml b/parm/jcb-gdas/atmosphere/model/posterior_output.yaml new file mode 100644 index 000000000..675d2ceaf --- /dev/null +++ b/parm/jcb-gdas/atmosphere/model/posterior_output.yaml @@ -0,0 +1,3 @@ +filetype: auxgrid +gridtype: gaussian +filename: ./mem%{member}%/atmanl. diff --git a/parm/jcb-gdas/atmosphere/observations/aircraft.yaml b/parm/jcb-gdas/atmosphere/observations/aircraft.yaml index 5b89dce7a..c9c94aa4e 100644 --- a/parm/jcb-gdas/atmosphere/observations/aircraft.yaml +++ b/parm/jcb-gdas/atmosphere/observations/aircraft.yaml @@ -328,3 +328,11 @@ threshold: 8.0 action: name: reject + +{% if algorithm == "test_obs_filters" %} + + geovals: + filename: aircraft_geoval_{{ current_cycle_YMDH }}.nc4 + passedBenchmark: 107747 + +{% endif %} diff --git a/parm/jcb-gdas/atmosphere/observations/amsua_n19.yaml b/parm/jcb-gdas/atmosphere/observations/amsua_n19.yaml index a4967c196..439145ec4 100644 --- a/parm/jcb-gdas/atmosphere/observations/amsua_n19.yaml +++ b/parm/jcb-gdas/atmosphere/observations/amsua_n19.yaml @@ -365,3 +365,11 @@ minvalue: 1.0e-12 action: name: reject + +{% if algorithm == "test_obs_filters" %} + + geovals: + filename: amsua_n19_geoval_{{ current_cycle_YMDH }}.nc4 + passedBenchmark: 107747 + +{% endif %} From e640beb0e0e5aa284713f3475eccd14e5c1e9876 Mon Sep 17 00:00:00 2001 From: danholdaway Date: Wed, 13 Mar 2024 16:16:17 -0400 Subject: [PATCH 008/199] reworking --- .../atmosphere/model/3dvar_outer_loop_1.yaml | 8 +- .../atmosphere/model/3dvar_outer_loop_2.yaml | 8 +- .../jcb-gdas/atmosphere/model/background.yaml | 15 +- .../atmosphere/model/background_ensemble.yaml | 19 +- .../background_error_hybrid_bump_bump.yaml | 19 +- .../background_error_hybrid_gsibec_bump.yaml | 24 +- .../model/background_error_static_bump.yaml | 2 +- .../model/background_error_static_gsibec.yaml | 7 +- .../background_error_static_identity.yaml | 2 +- .../atmosphere/model/final_increment.yaml | 17 - .../model/final_increment_gaussian.yaml | 17 + .../atmosphere/model/geometry_background.yaml | 8 +- .../atmosphere/model/model_pseudo.yaml | 5 +- .../model/output_ensemble_increments.yaml | 3 - .../output_ensemble_increments_gaussian.yaml | 3 + .../atmosphere/model/posterior_output.yaml | 3 - .../model/posterior_output_gaussian.yaml | 3 + .../atmosphere/observations/aircraft.yaml | 6 +- .../atmosphere/observations/amsua_n19.yaml | 120 +++--- .../observations/lgetkf_amsua_n19.yaml | 79 ++++ .../observations/lgetkf_conventional_ps.yaml | 357 ++++++++++++++++++ 21 files changed, 581 insertions(+), 144 deletions(-) delete mode 100644 parm/jcb-gdas/atmosphere/model/final_increment.yaml create mode 100644 parm/jcb-gdas/atmosphere/model/final_increment_gaussian.yaml delete mode 100644 parm/jcb-gdas/atmosphere/model/output_ensemble_increments.yaml create mode 100644 parm/jcb-gdas/atmosphere/model/output_ensemble_increments_gaussian.yaml delete mode 100644 parm/jcb-gdas/atmosphere/model/posterior_output.yaml create mode 100644 parm/jcb-gdas/atmosphere/model/posterior_output_gaussian.yaml create mode 100644 parm/jcb-gdas/atmosphere/observations/lgetkf_amsua_n19.yaml create mode 100644 parm/jcb-gdas/atmosphere/observations/lgetkf_conventional_ps.yaml diff --git a/parm/jcb-gdas/atmosphere/model/3dvar_outer_loop_1.yaml b/parm/jcb-gdas/atmosphere/model/3dvar_outer_loop_1.yaml index 566360652..e50d564c7 100644 --- a/parm/jcb-gdas/atmosphere/model/3dvar_outer_loop_1.yaml +++ b/parm/jcb-gdas/atmosphere/model/3dvar_outer_loop_1.yaml @@ -3,15 +3,15 @@ test: on geometry: fms initialization: - namelist filename: ./fv3jedi/fmsmpp.nml - field table filename: ./fv3jedi/field_table - akbk: ./fv3jedi/akbk.nc4 + namelist filename: {{fv3jedi_files_path}}/fmsmpp.nml + field table filename: {{fv3jedi_files_path}}/field_table + akbk: {{fv3jedi_files_path}}/akbk.nc4 layout: - {{layout_x}} - {{layout_y}} npx: {{npx_anl}} npy: {{npy_anl}} npz: {{npz_anl}} - field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_restart.yaml + field metadata override: {{fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml diagnostics: departures: bkgmob diff --git a/parm/jcb-gdas/atmosphere/model/3dvar_outer_loop_2.yaml b/parm/jcb-gdas/atmosphere/model/3dvar_outer_loop_2.yaml index 8044f7f9c..1c926f21c 100644 --- a/parm/jcb-gdas/atmosphere/model/3dvar_outer_loop_2.yaml +++ b/parm/jcb-gdas/atmosphere/model/3dvar_outer_loop_2.yaml @@ -3,15 +3,15 @@ test: on geometry: fms initialization: - namelist filename: ./fv3jedi/fmsmpp.nml - field table filename: ./fv3jedi/field_table - akbk: ./fv3jedi/akbk.nc4 + namelist filename: {{fv3jedi_files_path}}/fmsmpp.nml + field table filename: {{fv3jedi_files_path}}/field_table + akbk: {{fv3jedi_files_path}}/akbk.nc4 layout: - {{layout_x}} - {{layout_y}} npx: {{npx_anl}} npy: {{npy_anl}} npz: {{npz_anl}} - field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_restart.yaml + field metadata override: {{fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml diagnostics: departures: bkgmob1 diff --git a/parm/jcb-gdas/atmosphere/model/background.yaml b/parm/jcb-gdas/atmosphere/model/background.yaml index 936f61182..66d902073 100644 --- a/parm/jcb-gdas/atmosphere/model/background.yaml +++ b/parm/jcb-gdas/atmosphere/model/background.yaml @@ -1,11 +1,12 @@ -datapath: {{data}}/bkg +datapath: {{ background_path}} filetype: fms restart -datetime: '{{ atm_window_begin_isotime }}' -filename_core: '{{ atm_window_begin_fv3time }}.fv_core.res.nc' -filename_trcr: '{{ atm_window_begin_fv3time }}.fv_tracer.res.nc' -filename_sfcd: '{{ atm_window_begin_fv3time }}.sfc_data.nc' -filename_sfcw: '{{ atm_window_begin_fv3time }}.fv_srf_wnd.res.nc' -filename_cplr: '{{ atm_window_begin_fv3time }}.coupler.res' +datetime: "{{ atm_background_time_iso }}" +filename is datetime templated: true +filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" +filename_trcr: "%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc" +filename_sfcd: "%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc" +filename_sfcw: "%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc" +filename_cplr: "%yyyy%mm%dd.%hh%MM%ss.coupler.res" state variables: [ua,va,t,delp,sphum,ice_wat,liq_wat,o3mr,phis, slmsk,sheleg,tsea,vtype,stype,vfrac,stc,smc,snwdph, u_srf,v_srf,f10m] diff --git a/parm/jcb-gdas/atmosphere/model/background_ensemble.yaml b/parm/jcb-gdas/atmosphere/model/background_ensemble.yaml index 87a26ed54..9b3cf5909 100644 --- a/parm/jcb-gdas/atmosphere/model/background_ensemble.yaml +++ b/parm/jcb-gdas/atmosphere/model/background_ensemble.yaml @@ -1,17 +1,18 @@ -datetime: '{{ current_cycle_isotime }}' +datetime: '{{ atm_background_time_iso }}' members from template: template: - datetime: '{{ current_cycle_isotime }}' + datetime: '{{ atm_background_time_iso }}' filetype: fms restart state variables: [ua,va,t,DZ,delp,ps,sphum,ice_wat,liq_wat,o3mr,phis, slmsk,sheleg,tsea,vtype,stype,vfrac,stc,smc,snwdph, u_srf,v_srf,f10m] - datapath: bkg/mem%mem% - filename_core: '{{ current_cycle_fv3time }}.fv_core.res.nc' - filename_trcr: '{{ current_cycle_fv3time }}.fv_tracer.res.nc' - filename_sfcd: '{{ current_cycle_fv3time }}.sfc_data.nc' - filename_sfcw: '{{ current_cycle_fv3time }}.fv_srf_wnd.res.nc' - filename_cplr: '{{ current_cycle_fv3time }}.coupler.res' - pattern: '%mem%' + datapath: {{ background_ensemble_path}} + filename is datetime templated: true + filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" + filename_trcr: "%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc" + filename_sfcd: "%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc" + filename_sfcw: "%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc" + filename_cplr: "%yyyy%mm%dd.%hh%MM%ss.coupler.res" + pattern: "{{ ensemble_pattern | default("%mem%", true) }}" nmembers: {{ number_ensemble_members }} zero padding: 3 diff --git a/parm/jcb-gdas/atmosphere/model/background_error_hybrid_bump_bump.yaml b/parm/jcb-gdas/atmosphere/model/background_error_hybrid_bump_bump.yaml index 0c65415af..9d3f647df 100644 --- a/parm/jcb-gdas/atmosphere/model/background_error_hybrid_bump_bump.yaml +++ b/parm/jcb-gdas/atmosphere/model/background_error_hybrid_bump_bump.yaml @@ -10,7 +10,7 @@ components: active variables: &active_vars [psi,chi,t,ps,rh,liq_wat,o3mr] bump: io: - data directory: &staticb_dir {{data}}/berror + data directory: &staticb_dir {{bump_data_directory}} drivers: multivariate strategy: specific_univariate read local nicas: true @@ -95,16 +95,17 @@ components: covariance model: ensemble members from template: template: - datetime: '{{ current_cycle_isotime }}' + datetime: "{{ atm_background_time_iso }}" filetype: fms restart state variables: *3dvars - datapath: ens/mem%mem%/ - filename_core: '{{ current_cycle_fv3time }}.fv_core.res.nc' - filename_trcr: '{{ current_cycle_fv3time }}.fv_tracer.res.nc' - filename_sfcd: '{{ current_cycle_fv3time }}.sfc_data.nc' - filename_sfcw: '{{ current_cycle_fv3time }}.fv_srf_wnd.res.nc' - filename_cplr: '{{ current_cycle_fv3time }}.coupler.res' - pattern: '%mem%' + datapath: {{ background_ensemble_path}} + filename is datetime templated: true + filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" + filename_trcr: "%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc" + filename_sfcd: "%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc" + filename_sfcw: "%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc" + filename_cplr: "%yyyy%mm%dd.%hh%MM%ss.coupler.res" + pattern: "{{ ensemble_pattern | default("%mem%", true) }}" nmembers: {{number_ensemble_members}} zero padding: 3 # localization: diff --git a/parm/jcb-gdas/atmosphere/model/background_error_hybrid_gsibec_bump.yaml b/parm/jcb-gdas/atmosphere/model/background_error_hybrid_gsibec_bump.yaml index 926098a46..7802eaac4 100644 --- a/parm/jcb-gdas/atmosphere/model/background_error_hybrid_gsibec_bump.yaml +++ b/parm/jcb-gdas/atmosphere/model/background_error_hybrid_gsibec_bump.yaml @@ -5,10 +5,9 @@ components: saber central block: saber block name: gsi covariance read: - gsi akbk: &akbk {{data}}/fv3jedi/akbk.nc4 - gsi error covariance file: &gsiberr {{data}}/berror/gsi-coeffs-gfs-global.nc4 -# gsi error covariance file: &gsiberr {{data}}/berror/global_berror.f77 - gsi berror namelist file: &gsibnml {{data}}/berror/gfs_gsi_global.nml + gsi akbk: &akbk {{fv3jedi_files_path}}/akbk.nc4 + gsi error covariance file: &gsiberr {{gsibec_path}}/gsi-coeffs-gfs-global.nc4 + gsi berror namelist file: &gsibnml {{gsibec_path}}/gfs_gsi_global.nml processor layout x direction: &layout_gsib_x {{layout_gsib_x}} processor layout y direction: &layout_gsib_y {{layout_gsib_y}} debugging mode: false @@ -32,16 +31,17 @@ components: covariance model: ensemble members from template: template: - datetime: '{{ current_cycle_isotime }}' + datetime: "{{ atm_background_time_iso }}" filetype: fms restart state variables: *3dvars_anal - datapath: ens/mem%mem% - filename_core: '{{ current_cycle_fv3time }}.fv_core.res.nc' - filename_trcr: '{{ current_cycle_fv3time }}.fv_tracer.res.nc' - filename_sfcd: '{{ current_cycle_fv3time }}.sfc_data.nc' - filename_sfcw: '{{ current_cycle_fv3time }}.fv_srf_wnd.res.nc' - filename_cplr: '{{ current_cycle_fv3time }}.coupler.res' - pattern: '%mem%' + datapath: {{ background_ensemble_path}} + filename is datetime templated: true + filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" + filename_trcr: "%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc" + filename_sfcd: "%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc" + filename_sfcw: "%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc" + filename_cplr: "%yyyy%mm%dd.%hh%MM%ss.coupler.res" + pattern: "{{ ensemble_pattern | default("%mem%", true) }}" nmembers: {{number_ensemble_members}} zero padding: 3 localization: diff --git a/parm/jcb-gdas/atmosphere/model/background_error_static_bump.yaml b/parm/jcb-gdas/atmosphere/model/background_error_static_bump.yaml index 9dc16f7cf..65be183d6 100644 --- a/parm/jcb-gdas/atmosphere/model/background_error_static_bump.yaml +++ b/parm/jcb-gdas/atmosphere/model/background_error_static_bump.yaml @@ -8,7 +8,7 @@ saber blocks: active variables: &active_vars [psi,chi,t,ps,rh,liq_wat,o3mr] bump: io: - data directory: &staticb_dir {{data}}/berror + data directory: &staticb_dir {{bump_data_directory}} drivers: multivariate strategy: specific_univariate read local nicas: true diff --git a/parm/jcb-gdas/atmosphere/model/background_error_static_gsibec.yaml b/parm/jcb-gdas/atmosphere/model/background_error_static_gsibec.yaml index 2d8d01a8d..86a863302 100644 --- a/parm/jcb-gdas/atmosphere/model/background_error_static_gsibec.yaml +++ b/parm/jcb-gdas/atmosphere/model/background_error_static_gsibec.yaml @@ -3,10 +3,9 @@ full inverse: true saber central block: saber block name: gsi covariance read: - gsi akbk: &akbk {{data}}/fv3jedi/akbk.nc4 - gsi error covariance file: &gsiberr {{data}}/berror/gsi-coeffs-gfs-global.nc4 -# gsi error covariance file: &gsiberr {{data}}/berror/global_berror.f77 - gsi berror namelist file: &gsibnml {{data}}/berror/gfs_gsi_global.nml + gsi akbk: &akbk {{fv3jedi_files_path}}/akbk.nc4 + gsi error covariance file: &gsiberr {{gsibec_path}}/gsi-coeffs-gfs-global.nc4 + gsi berror namelist file: &gsibnml {{gsibec_path}}/gfs_gsi_global.nml processor layout x direction: &layout_gsib_x {{layout_gsib_x}} processor layout y direction: &layout_gsib_y {{layout_gsib_y}} debugging mode: false diff --git a/parm/jcb-gdas/atmosphere/model/background_error_static_identity.yaml b/parm/jcb-gdas/atmosphere/model/background_error_static_identity.yaml index 62fc984c6..c5e6c99d8 100644 --- a/parm/jcb-gdas/atmosphere/model/background_error_static_identity.yaml +++ b/parm/jcb-gdas/atmosphere/model/background_error_static_identity.yaml @@ -1,2 +1,2 @@ covariance model: FV3JEDI-ID -date: '{{ATM_WINDOW_BEGIN}}' +date: "{{ atm_background_time_iso }}" diff --git a/parm/jcb-gdas/atmosphere/model/final_increment.yaml b/parm/jcb-gdas/atmosphere/model/final_increment.yaml deleted file mode 100644 index 901fed375..000000000 --- a/parm/jcb-gdas/atmosphere/model/final_increment.yaml +++ /dev/null @@ -1,17 +0,0 @@ -output: - state component: - filetype: auxgrid - gridtype: gaussian - filename: ./anl/atminc. -geometry: - fms initialization: - namelist filename: ./fv3jedi/fmsmpp.nml - field table filename: ./fv3jedi/field_table - akbk: ./fv3jedi/akbk.nc4 - layout: - - {{layout_x}} - - {{layout_y}} - npx: {{npx_anl}} - npy: {{npy_anl}} - npz: {{npz_anl}} - field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_restart.yaml diff --git a/parm/jcb-gdas/atmosphere/model/final_increment_gaussian.yaml b/parm/jcb-gdas/atmosphere/model/final_increment_gaussian.yaml new file mode 100644 index 000000000..a57243361 --- /dev/null +++ b/parm/jcb-gdas/atmosphere/model/final_increment_gaussian.yaml @@ -0,0 +1,17 @@ +output: + state component: + filetype: auxgrid + gridtype: gaussian + filename: {{final_increment_prefix}} +geometry: + fms initialization: + namelist filename: "{{fv3jedi_files_path}}/fmsmpp.nml" + field table filename: "{{fv3jedi_files_path}}/field_table" + akbk: "{{fv3jedi_files_path}}/akbk.nc4" + layout: + - {{layout_x}} + - {{layout_y}} + npx: {{npx_anl}} + npy: {{npy_anl}} + npz: {{npz_anl}} + field metadata override: "{{fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" diff --git a/parm/jcb-gdas/atmosphere/model/geometry_background.yaml b/parm/jcb-gdas/atmosphere/model/geometry_background.yaml index ba5789089..fefe71c1b 100644 --- a/parm/jcb-gdas/atmosphere/model/geometry_background.yaml +++ b/parm/jcb-gdas/atmosphere/model/geometry_background.yaml @@ -1,11 +1,11 @@ fms initialization: - namelist filename: {{data}}/fv3jedi/fmsmpp.nml - field table filename: {{data}}/fv3jedi/field_table -akbk: {{data}}/fv3jedi/akbk.nc4 + namelist filename: "{{fv3jedi_files_path}}/fmsmpp.nml" + field table filename: "{{fv3jedi_files_path}}/field_table" +akbk: "{{fv3jedi_files_path}}/akbk.nc4" layout: - {{layout_x}} - {{layout_y}} npx: {{npx_ges}} npy: {{npy_ges}} npz: {{npz_ges}} -field metadata override: {{data}}/fv3jedi/fv3jedi_fieldmetadata_restart.yaml +field metadata override: "{{fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" diff --git a/parm/jcb-gdas/atmosphere/model/model_pseudo.yaml b/parm/jcb-gdas/atmosphere/model/model_pseudo.yaml index 93d3ab7c2..05eefc26c 100644 --- a/parm/jcb-gdas/atmosphere/model/model_pseudo.yaml +++ b/parm/jcb-gdas/atmosphere/model/model_pseudo.yaml @@ -1,14 +1,13 @@ name: PSEUDO -datapath: {{data}}/bkg +datapath: {{ background_path}} filetype: fms restart -datetime: '{{atm_window_begin_isotime}}' filename is datetime templated: true filename_core: '%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc' filename_trcr: '%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc' filename_sfcd: '%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc' filename_sfcw: '%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc' filename_cplr: '%yyyy%mm%dd.%hh%MM%ss.coupler.res' -tstep: {{bkg_tstep}} +tstep: {{forecast_timestep}} model variables: [ua,va,t,delp,sphum,ice_wat,liq_wat,o3mr,phis, slmsk,sheleg,tsea,vtype,stype,vfrac,stc,smc,snwdph, u_srf,v_srf,f10m] diff --git a/parm/jcb-gdas/atmosphere/model/output_ensemble_increments.yaml b/parm/jcb-gdas/atmosphere/model/output_ensemble_increments.yaml deleted file mode 100644 index 7f9952872..000000000 --- a/parm/jcb-gdas/atmosphere/model/output_ensemble_increments.yaml +++ /dev/null @@ -1,3 +0,0 @@ -filetype: auxgrid -gridtype: gaussian -filename: ./anl/mem%{member}%/atminc. diff --git a/parm/jcb-gdas/atmosphere/model/output_ensemble_increments_gaussian.yaml b/parm/jcb-gdas/atmosphere/model/output_ensemble_increments_gaussian.yaml new file mode 100644 index 000000000..d11ca4db2 --- /dev/null +++ b/parm/jcb-gdas/atmosphere/model/output_ensemble_increments_gaussian.yaml @@ -0,0 +1,3 @@ +filetype: auxgrid +gridtype: gaussian +filename: {{ensemble_increment_prefix}} diff --git a/parm/jcb-gdas/atmosphere/model/posterior_output.yaml b/parm/jcb-gdas/atmosphere/model/posterior_output.yaml deleted file mode 100644 index 675d2ceaf..000000000 --- a/parm/jcb-gdas/atmosphere/model/posterior_output.yaml +++ /dev/null @@ -1,3 +0,0 @@ -filetype: auxgrid -gridtype: gaussian -filename: ./mem%{member}%/atmanl. diff --git a/parm/jcb-gdas/atmosphere/model/posterior_output_gaussian.yaml b/parm/jcb-gdas/atmosphere/model/posterior_output_gaussian.yaml new file mode 100644 index 000000000..471137f4a --- /dev/null +++ b/parm/jcb-gdas/atmosphere/model/posterior_output_gaussian.yaml @@ -0,0 +1,3 @@ +filetype: auxgrid +gridtype: gaussian +filename: {{posterior_output_gaussian}} diff --git a/parm/jcb-gdas/atmosphere/observations/aircraft.yaml b/parm/jcb-gdas/atmosphere/observations/aircraft.yaml index c9c94aa4e..cc957e662 100644 --- a/parm/jcb-gdas/atmosphere/observations/aircraft.yaml +++ b/parm/jcb-gdas/atmosphere/observations/aircraft.yaml @@ -3,7 +3,7 @@ obsdatain: engine: type: H5File - obsfile: {{data}}/obs/{{oprefix}}aircraft.{{ current_cycle_YMDH }}.nc + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" obsgrouping: group variables: ["stationIdentification"] sort variable: "pressure" @@ -11,7 +11,7 @@ obsdataout: engine: type: H5File - obsfile: {{data}}/diags/diag_aircraft_{{ current_cycle_YMDH }}.nc + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [windEastward, windNorthward, airTemperature, specificHumidity] @@ -332,7 +332,7 @@ {% if algorithm == "test_obs_filters" %} geovals: - filename: aircraft_geoval_{{ current_cycle_YMDH }}.nc4 + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" passedBenchmark: 107747 {% endif %} diff --git a/parm/jcb-gdas/atmosphere/observations/amsua_n19.yaml b/parm/jcb-gdas/atmosphere/observations/amsua_n19.yaml index 439145ec4..f7ab997e4 100644 --- a/parm/jcb-gdas/atmosphere/observations/amsua_n19.yaml +++ b/parm/jcb-gdas/atmosphere/observations/amsua_n19.yaml @@ -1,37 +1,37 @@ - obs space: - name: amsua_n19 + name: AMSU-A NOAA-19 obsdatain: engine: type: H5File - obsfile: {{data}}/obs/{{oprefix}}amsua_n19.{{ current_cycle_YMDH }}.nc + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: {{data}}/diags/diag_amsua_n19_{{ current_cycle_YMDH }}.nc + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [brightnessTemperature] - channels: &amsua_n19_channels 1-15 + channels: &{{observation_from_jcb}}_channels 1-15 obs operator: name: CRTM Absorbers: [H2O,O3] Clouds: [Water, Ice] Cloud_Fraction: 1.0 obs options: - Sensor_ID: amsua_n19 + Sensor_ID: {{observation_from_jcb}} EndianType: little_endian - CoefficientPath: {{data}}/crtm/ + CoefficientPath: {{crtm_coefficient_path}} obs bias: - input file: {{data}}/obs/{{gprefix}}amsua_n19.satbias.nc - output file: {{data}}/bc/{{aprefix}}amsua_n19.satbias.nc + input file: "{{atm_obsbiasin_path}}/{{atm_obsbiasin_prefix}}{{observation_from_jcb}}{{atm_obsbiasin_suffix}}" + output file: "{{atm_obsbiasout_path}}/{{atm_obsbiasout_prefix}}{{observation_from_jcb}}{{atm_obsbiasout_suffix}}" variational bc: predictors: - name: constant - name: lapse_rate order: 2 - tlapse: &amsua_n19_tlapse {{data}}/obs/{{gprefix}}amsua_n19.tlapse.txt + tlapse: &{{observation_from_jcb}}_tlapse "{{atm_obsbiasin_path}}/{{atm_obstlapsein_prefix}}{{observation_from_jcb}}{{atm_obstlapsein_suffix}}" - name: lapse_rate - tlapse: *amsua_n19_tlapse + tlapse: *{{observation_from_jcb}}_tlapse - name: emissivity - name: scan_angle order: 4 @@ -46,23 +46,23 @@ step size: 1.0e-4 largest analysis variance: 10000.0 prior: - input file: {{data}}/obs/{{gprefix}}amsua_n19.satbias_cov.nc + input file: "{{atm_obsbiasin_path}}/{{atm_obsbiascovin_prefix}}{{observation_from_jcb}}{{atm_obsbiascovin_suffix}}" inflation: ratio: 1.1 ratio for small dataset: 2.0 - output file: {{data}}/bc/{{aprefix}}amsua_n19.satbias_cov.nc + output file: "{{atm_obsbiasout_path}}/{{atm_obsbiascovout_prefix}}{{observation_from_jcb}}{{atm_obsbiascovout_suffix}}" obs filters: - filter: BlackList filter variables: - name: brightnessTemperature - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels action: name: assign error error function: name: ObsFunction/ObsErrorModelRamp - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels options: - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels xvar: name: ObsFunction/CLWRetSymmetricMW options: @@ -113,12 +113,12 @@ - filter: Bounds Check filter variables: - name: brightnessTemperature - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels test variables: - name: ObsFunction/HydrometeorCheckAMSUA - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels options: - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels obserr_clearsky: [ 2.500, 2.200, 2.000, 0.550, 0.300, 0.230, 0.230, 0.250, 0.250, 0.350, 0.400, 0.550, 0.800, 3.000, 3.500] @@ -130,9 +130,9 @@ clwret_types: [ObsValue] obserr_function: name: ObsFunction/ObsErrorModelRamp - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels options: - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels xvar: name: ObsFunction/CLWRetSymmetricMW options: @@ -158,56 +158,56 @@ - filter: BlackList filter variables: - name: brightnessTemperature - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels action: name: inflate error inflation variable: name: ObsFunction/ObsErrorFactorTopoRad - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels options: - sensor: amsua_n19 - channels: *amsua_n19_channels + sensor: {{observation_from_jcb}} + channels: *{{observation_from_jcb}}_channels # Transmittnace Top Check - filter: BlackList filter variables: - name: brightnessTemperature - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels action: name: inflate error inflation variable: name: ObsFunction/ObsErrorFactorTransmitTopRad - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels options: - sensor: amsua_n19 - channels: *amsua_n19_channels + sensor: {{observation_from_jcb}} + channels: *{{observation_from_jcb}}_channels # Surface Jacobian check - filter: BlackList filter variables: - name: brightnessTemperature - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels action: name: inflate error inflation variable: name: ObsFunction/ObsErrorFactorSurfJacobianRad - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels options: - sensor: amsua_n19 - channels: *amsua_n19_channels + sensor: {{observation_from_jcb}} + channels: *{{observation_from_jcb}}_channels obserr_demisf: [0.010, 0.020, 0.015, 0.020, 0.200] obserr_dtempf: [0.500, 2.000, 1.000, 2.000, 4.500] # Situation dependent Check - filter: BlackList filter variables: - name: brightnessTemperature - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels action: name: inflate error inflation variable: name: ObsFunction/ObsErrorFactorSituDependMW - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels options: - sensor: amsua_n19 - channels: *amsua_n19_channels + sensor: {{observation_from_jcb}} + channels: *{{observation_from_jcb}}_channels clwobs_function: name: ObsFunction/CLWRetMW options: @@ -231,9 +231,9 @@ bias_application: HofX clwmatchidx_function: name: ObsFunction/CLWMatchIndexMW - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels options: - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels clwobs_function: name: ObsFunction/CLWRetMW options: @@ -252,9 +252,9 @@ 0.000, 0.000, 0.000, 0.000, 0.030] obserr_function: name: ObsFunction/ObsErrorModelRamp - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels options: - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels xvar: name: ObsFunction/CLWRetSymmetricMW options: @@ -280,33 +280,33 @@ - filter: Background Check filter variables: - name: brightnessTemperature - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels function absolute threshold: - name: ObsFunction/ObsErrorBoundMW - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels options: - sensor: amsua_n19 - channels: *amsua_n19_channels + sensor: {{observation_from_jcb}} + channels: *{{observation_from_jcb}}_channels obserr_bound_latitude: name: ObsFunction/ObsErrorFactorLatRad options: latitude_parameters: [25.0, 0.25, 0.04, 3.0] obserr_bound_transmittop: name: ObsFunction/ObsErrorFactorTransmitTopRad - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels options: - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels obserr_bound_topo: name: ObsFunction/ObsErrorFactorTopoRad - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels options: - channels: *amsua_n19_channels - sensor: amsua_n19 + channels: *{{observation_from_jcb}}_channels + sensor: {{observation_from_jcb}} obserr_function: name: ObsFunction/ObsErrorModelRamp - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels options: - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels xvar: name: ObsFunction/CLWRetSymmetricMW options: @@ -335,13 +335,13 @@ - filter: Bounds Check filter variables: - name: brightnessTemperature - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels test variables: - name: ObsFunction/InterChannelConsistencyCheck - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels options: - channels: *amsua_n19_channels - sensor: amsua_n19 + channels: *{{observation_from_jcb}}_channels + sensor: {{observation_from_jcb}} use_flag: [ 1, 1, 1, 1, 1, 1, -1, -1, 1, 1, 1, 1, 1, -1, 1 ] @@ -352,13 +352,13 @@ - filter: Bounds Check filter variables: - name: brightnessTemperature - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels test variables: - name: ObsFunction/ChannelUseflagCheckRad - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels options: - sensor: amsua_n19 - channels: *amsua_n19_channels + sensor: {{observation_from_jcb}} + channels: *{{observation_from_jcb}}_channels use_flag: [ 1, 1, 1, 1, 1, 1, -1, -1, 1, 1, 1, 1, 1, -1, 1 ] @@ -369,7 +369,7 @@ {% if algorithm == "test_obs_filters" %} geovals: - filename: amsua_n19_geoval_{{ current_cycle_YMDH }}.nc4 + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" passedBenchmark: 107747 {% endif %} diff --git a/parm/jcb-gdas/atmosphere/observations/lgetkf_amsua_n19.yaml b/parm/jcb-gdas/atmosphere/observations/lgetkf_amsua_n19.yaml new file mode 100644 index 000000000..fe4c603c4 --- /dev/null +++ b/parm/jcb-gdas/atmosphere/observations/lgetkf_amsua_n19.yaml @@ -0,0 +1,79 @@ +- obs space: + name: AMSU-A NOAA-19 + distribution: + name: Halo + halo size: 1250e3 + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}amsua_n19{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: './diags/diag_amsua_n19_{{ current_cycle | to_YMDH }}.nc' + io pool: + max pool size: 1 + simulated variables: [brightnessTemperature] + channels: &amsua_n19_channels 4-6,9-14 + obs operator: + name: CRTM + Absorbers: [H2O,O3] + Clouds: [Water, Ice] + Cloud_Fraction: 1.0 + obs options: + Sensor_ID: amsua_n19 + EndianType: little_endian + CoefficientPath: ./crtm/ + obs error: + covariance model: diagonal + obs bias: + input file: "{{atm_obsbiasin_path}}/{{atm_obsbiasin_prefix}}amsua_n19{{atm_obsbiasin_suffix}}" + output file: "{{atm_obsbiasout_path}}/{{atm_obsbiasout_prefix}}amsua_n19{{atm_obsbiasout_suffix}}" + variational bc: + predictors: + - name: constant + - name: lapse_rate + order: 2 + tlapse: &amsua_n19_tlapse "{{atm_obsbiasin_path}}/{{atm_obstlapsein_prefix}}amsua_n19{{atm_obstlapsein_suffix}}" + - name: lapse_rate + tlapse: *amsua_n19_tlapse + - name: emissivity + - name: scan_angle + order: 4 + - name: scan_angle + order: 3 + - name: scan_angle + order: 2 + - name: scan_angle + covariance: + minimal required obs number: 20 + variance range: [1.0e-6, 10.0] + step size: 1.0e-4 + largest analysis variance: 10000.0 + prior: + input file: "{{atm_obsbiasin_path}}/{{atm_obsbiascovin_prefix}}amsua_n19{{atm_obsbiascovin_suffix}}" + inflation: + ratio: 1.1 + ratio for small dataset: 2.0 + output file: "{{atm_obsbiasout_path}}/{{atm_obsbiascovout_prefix}}amsua_n19{{atm_obsbiascovout_suffix}}" + obs filters: + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *amsua_n19_channels + minvalue: 100.0 + maxvalue: 500.0 + action: + name: reject + # Gross check + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: *amsua_n19_channels + threshold: 3.0 + action: + name: reject + obs localizations: + - localization method: Horizontal Gaspari-Cohn + lengthscale: 1250e3 + max nobs: 10000 diff --git a/parm/jcb-gdas/atmosphere/observations/lgetkf_conventional_ps.yaml b/parm/jcb-gdas/atmosphere/observations/lgetkf_conventional_ps.yaml new file mode 100644 index 000000000..170baaba6 --- /dev/null +++ b/parm/jcb-gdas/atmosphere/observations/lgetkf_conventional_ps.yaml @@ -0,0 +1,357 @@ +- obs space: + name: surface_ps + distribution: + name: Halo + halo size: 1250e3 + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}conventional_ps{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: '{{ DATA }}/diags/diag_conventional_ps_prepbufr_{{ current_cycle | to_YMDH }}.nc' + io pool: + max pool size: 1 + simulated variables: [stationPressure] + + obs operator: + name: SfcPCorrected + variables: + - name: stationPressure + da_psfc_scheme: GSI + station_altitude: height + geovar_sfc_geomz: surface_altitude + geovar_geomz: geopotential_height + + linear obs operator: + name: Identity + variables: + - name: stationPressure + + obs prior filters: + # Initial Error Assignments for SFC Observations + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [181] + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + round_to_the_nearest_integer: true + xvar: + name: ObsValue/stationPressure + xvals: [80000, 75000, 70000, 65000, 60000, 55000 ] + errors: [110, 120, 120, 120, 120, 1.0e+11] + + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [187] + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + round_to_the_nearest_integer: true + xvar: + name: ObsValue/stationPressure + xvals: [85000, 80000, 75000, 70000, 65000, 60000, 55000 ] + errors: [ 120, 140, 140, 140, 140, 140, 1.0e+11] + + # Initial Error Assignments for SFCSHIP Observations + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [180] + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + round_to_the_nearest_integer: true + xvar: + name: ObsValue/stationPressure + xvals: [60000, 55000 ] + errors: [ 130, 1.0e+11] + + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [183] + action: + name: assign error + error parameter: 1.0e+11 + + # Initial Error Assignments for Radiosonde + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [120] + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + round_to_the_nearest_integer: true + xvar: + name: ObsValue/stationPressure + xvals: [80000, 75000, 70000, 65000, 60000, 55000 ] + errors: [ 110, 120, 120, 120, 120, 1.0e+11] + + # Create PreQC group variable (pqm in GSI read_prepbufr) + - filter: Variable Assignment + assignments: + - name: InputObsError/stationPressure + type: float + source variable: ObsErrorData/stationPressure + + # Set observation quality-realted variables + # Create PreQC group variable (pqm in GSI read_prepbufr) + - filter: Variable Assignment + assignments: + - name: PreQC/stationPressure + type: int + source variable: QualityMarker/stationPressure + + # Create PreUseFlag group variable (usage in GSI read_prepbufr) + # Initialize + - filter: Variable Assignment + assignments: + - name: PreUseFlag/stationPressure + type: int + source variable: PreQC/stationPressure + + - filter: Variable Assignment + where: + - variable: + name: PreUseFlag/stationPressure + is_in: 1-15 + assignments: + - name: PreUseFlag/stationPressure + value: 0 + # Re-assignment + - filter: Variable Assignment + where: + - variable: + name: ObsType/stationPressure + is_in: 183 + assignments: + - name: PreUseFlag/stationPressure + value: 100 + + - filter: Variable Assignment + where: + - variable: + name: ObsValue/stationPressure + is_defined: + - variable: + name: ObsValue/stationPressure + maxvalue: 50000.00 + where operator: and + assignments: + - name: PreUseFlag/stationPressure + value: 100 + + - filter: Variable Assignment + where: + - variable: + name: PreQC/stationPressure + is_in: 9, 12, 15 + assignments: + - name: PreUseFlag/stationPressure + value: 100 + + - filter: Variable Assignment + where: + - variable: + name: PreQC/stationPressure + is_in: 4-15 + assignments: + - name: PreUseFlag/stationPressure + value: 101 + + ## Observation range sanity check + #- filter: Bounds Check + # filter variables: + # - name: stationPressure + # minvalue: 37499.0 + # maxvalue: 106999.0 + # action: + # name: reject + + ## Reject all ObsType 183 + #- filter: RejectList + # where: + # - variable: + # name: ObsType/stationPressure + # is_in: 183 + + ## Reject surface pressure below 500 hPa + #- filter: Bounds Check + # filter variables: + # - name: stationPressure + # minvalue: 50000.00 + # action: + # name: reject + + #- filter: RejectList + # where: + # - variable: + # name: PreQC/stationPressure + # is_in: 4-15 + + # Inflate obs error based on obs type + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: PreQC/stationPressure + is_in: 3, 7 + action: + name: inflate error + inflation factor: 1.2 + + obs post filters: + # Calculate obs error inflation factors for duplicated observations at the same location + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorDuplicateCheck/stationPressure + type: float + function: + name: ObsFunction/ObsErrorFactorDuplicateCheck + options: + use_air_pressure: false + variable: stationPressure + + # Reduce effective observation error based on obs type and subtype + # In this case: reduce effective obs error for buoy + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: + name: ObsType/stationPressure + is_in: 180 + - variable: + name: ObsSubType/stationPressure + is_in: 0 + action: + name: inflate error + inflation factor: 0.7 + + # Calculate obs error inflation factors for large discrepancies between model and observations + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorSfcPressure/stationPressure + type: float + function: + name: ObsFunction/ObsErrorFactorSfcPressure + options: + geovar_sfc_geomz: surface_altitude + geovar_geomz: geopotential_height + station_altitude: height + + # Inflate surface pressure observation based on discrepancies between + # model and observations due to terrian + - filter: Perform Action + filter variables: + - name: stationPressure + action: + name: inflate error + inflation variable: + name: ObsErrorFactorSfcPressure/stationPressure + + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/Innovation + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsValue/stationPressure + - name: HofX/stationPressure + coefs: [1, -1] + + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ObsErrorBoundSfcPressure1 + type: float + function: + name: ObsFunction/ObsErrorBoundConventional + options: + obsvar: stationPressure + obserr_bound_min: 100 + obserr_bound_max: 300 + obserr_bound_factor: 5.0 + + - filter: Background Check + filter variables: + - name: stationPressure + where: + - variable: PreQC/stationPressure + is_not_in: 3 + function absolute threshold: + - name: DerivedMetaData/ObsErrorBoundSfcPressure1 + action: + name: reject + + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ObsErrorBoundSfcPressure2 + type: float + function: + name: ObsFunction/ObsErrorBoundConventional + options: + obsvar: stationPressure + obserr_bound_min: 100 + obserr_bound_max: 300 + obserr_bound_factor: 3.5 + + - filter: Background Check + filter variables: + - name: stationPressure + where: + - variable: PreQC/stationPressure + is_in: 3 + function absolute threshold: + - name: DerivedMetaData/ObsErrorBoundSfcPressure2 + action: + name: reject + + # Inflate obs error based on duplicate check + - filter: Perform Action + filter variables: + - name: stationPressure + action: + name: inflate error + inflation variable: + name: ObsErrorFactorDuplicateCheck/stationPressure + + # Reject data based on PreUseFlag (usage in GSI) + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: PreUseFlag/stationPressure + is_not_in: 0, 1 + action: + name: reject + # End of Filters + obs localizations: + - localization method: Horizontal Gaspari-Cohn + lengthscale: 1250e3 + max nobs: 10000 From 2e0f57d5044dbc217b23da35fec14275845801c2 Mon Sep 17 00:00:00 2001 From: danholdaway Date: Wed, 13 Mar 2024 16:23:52 -0400 Subject: [PATCH 009/199] reorder directory structure --- parm/jcb-gdas/{aero/model => model/aero}/geometry.yaml | 0 .../model => model/atmosphere}/3dvar_outer_loop_1.yaml | 0 .../model => model/atmosphere}/3dvar_outer_loop_2.yaml | 0 .../{atmosphere/model => model/atmosphere}/background.yaml | 0 .../model => model/atmosphere}/background_ensemble.yaml | 0 .../atmosphere}/background_error_hybrid_bump_bump.yaml | 0 .../atmosphere}/background_error_hybrid_gsibec_bump.yaml | 0 .../model => model/atmosphere}/background_error_static_bump.yaml | 0 .../atmosphere}/background_error_static_gsibec.yaml | 0 .../atmosphere}/background_error_static_identity.yaml | 0 .../model => model/atmosphere}/final_increment_gaussian.yaml | 0 .../model => model/atmosphere}/geometry_background.yaml | 0 .../{atmosphere/model => model/atmosphere}/model_pseudo.yaml | 0 .../atmosphere}/output_ensemble_increments_gaussian.yaml | 0 .../model => model/atmosphere}/posterior_output_gaussian.yaml | 0 parm/jcb-gdas/{marine/model => model/marine}/geometry.yaml | 0 parm/jcb-gdas/{snow/model => model/snow}/geometry.yaml | 0 parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml | 1 + .../atmosphere-lgetkf/amsua_n19.yaml} | 0 .../atmosphere-lgetkf/conventional_ps.yaml} | 0 .../observations => observations/atmosphere}/aircraft.yaml | 0 .../observations => observations/atmosphere}/amsua_n19.yaml | 0 22 files changed, 1 insertion(+) rename parm/jcb-gdas/{aero/model => model/aero}/geometry.yaml (100%) rename parm/jcb-gdas/{atmosphere/model => model/atmosphere}/3dvar_outer_loop_1.yaml (100%) rename parm/jcb-gdas/{atmosphere/model => model/atmosphere}/3dvar_outer_loop_2.yaml (100%) rename parm/jcb-gdas/{atmosphere/model => model/atmosphere}/background.yaml (100%) rename parm/jcb-gdas/{atmosphere/model => model/atmosphere}/background_ensemble.yaml (100%) rename parm/jcb-gdas/{atmosphere/model => model/atmosphere}/background_error_hybrid_bump_bump.yaml (100%) rename parm/jcb-gdas/{atmosphere/model => model/atmosphere}/background_error_hybrid_gsibec_bump.yaml (100%) rename parm/jcb-gdas/{atmosphere/model => model/atmosphere}/background_error_static_bump.yaml (100%) rename parm/jcb-gdas/{atmosphere/model => model/atmosphere}/background_error_static_gsibec.yaml (100%) rename parm/jcb-gdas/{atmosphere/model => model/atmosphere}/background_error_static_identity.yaml (100%) rename parm/jcb-gdas/{atmosphere/model => model/atmosphere}/final_increment_gaussian.yaml (100%) rename parm/jcb-gdas/{atmosphere/model => model/atmosphere}/geometry_background.yaml (100%) rename parm/jcb-gdas/{atmosphere/model => model/atmosphere}/model_pseudo.yaml (100%) rename parm/jcb-gdas/{atmosphere/model => model/atmosphere}/output_ensemble_increments_gaussian.yaml (100%) rename parm/jcb-gdas/{atmosphere/model => model/atmosphere}/posterior_output_gaussian.yaml (100%) rename parm/jcb-gdas/{marine/model => model/marine}/geometry.yaml (100%) rename parm/jcb-gdas/{snow/model => model/snow}/geometry.yaml (100%) create mode 120000 parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml rename parm/jcb-gdas/{atmosphere/observations/lgetkf_amsua_n19.yaml => observations/atmosphere-lgetkf/amsua_n19.yaml} (100%) rename parm/jcb-gdas/{atmosphere/observations/lgetkf_conventional_ps.yaml => observations/atmosphere-lgetkf/conventional_ps.yaml} (100%) rename parm/jcb-gdas/{atmosphere/observations => observations/atmosphere}/aircraft.yaml (100%) rename parm/jcb-gdas/{atmosphere/observations => observations/atmosphere}/amsua_n19.yaml (100%) diff --git a/parm/jcb-gdas/aero/model/geometry.yaml b/parm/jcb-gdas/model/aero/geometry.yaml similarity index 100% rename from parm/jcb-gdas/aero/model/geometry.yaml rename to parm/jcb-gdas/model/aero/geometry.yaml diff --git a/parm/jcb-gdas/atmosphere/model/3dvar_outer_loop_1.yaml b/parm/jcb-gdas/model/atmosphere/3dvar_outer_loop_1.yaml similarity index 100% rename from parm/jcb-gdas/atmosphere/model/3dvar_outer_loop_1.yaml rename to parm/jcb-gdas/model/atmosphere/3dvar_outer_loop_1.yaml diff --git a/parm/jcb-gdas/atmosphere/model/3dvar_outer_loop_2.yaml b/parm/jcb-gdas/model/atmosphere/3dvar_outer_loop_2.yaml similarity index 100% rename from parm/jcb-gdas/atmosphere/model/3dvar_outer_loop_2.yaml rename to parm/jcb-gdas/model/atmosphere/3dvar_outer_loop_2.yaml diff --git a/parm/jcb-gdas/atmosphere/model/background.yaml b/parm/jcb-gdas/model/atmosphere/background.yaml similarity index 100% rename from parm/jcb-gdas/atmosphere/model/background.yaml rename to parm/jcb-gdas/model/atmosphere/background.yaml diff --git a/parm/jcb-gdas/atmosphere/model/background_ensemble.yaml b/parm/jcb-gdas/model/atmosphere/background_ensemble.yaml similarity index 100% rename from parm/jcb-gdas/atmosphere/model/background_ensemble.yaml rename to parm/jcb-gdas/model/atmosphere/background_ensemble.yaml diff --git a/parm/jcb-gdas/atmosphere/model/background_error_hybrid_bump_bump.yaml b/parm/jcb-gdas/model/atmosphere/background_error_hybrid_bump_bump.yaml similarity index 100% rename from parm/jcb-gdas/atmosphere/model/background_error_hybrid_bump_bump.yaml rename to parm/jcb-gdas/model/atmosphere/background_error_hybrid_bump_bump.yaml diff --git a/parm/jcb-gdas/atmosphere/model/background_error_hybrid_gsibec_bump.yaml b/parm/jcb-gdas/model/atmosphere/background_error_hybrid_gsibec_bump.yaml similarity index 100% rename from parm/jcb-gdas/atmosphere/model/background_error_hybrid_gsibec_bump.yaml rename to parm/jcb-gdas/model/atmosphere/background_error_hybrid_gsibec_bump.yaml diff --git a/parm/jcb-gdas/atmosphere/model/background_error_static_bump.yaml b/parm/jcb-gdas/model/atmosphere/background_error_static_bump.yaml similarity index 100% rename from parm/jcb-gdas/atmosphere/model/background_error_static_bump.yaml rename to parm/jcb-gdas/model/atmosphere/background_error_static_bump.yaml diff --git a/parm/jcb-gdas/atmosphere/model/background_error_static_gsibec.yaml b/parm/jcb-gdas/model/atmosphere/background_error_static_gsibec.yaml similarity index 100% rename from parm/jcb-gdas/atmosphere/model/background_error_static_gsibec.yaml rename to parm/jcb-gdas/model/atmosphere/background_error_static_gsibec.yaml diff --git a/parm/jcb-gdas/atmosphere/model/background_error_static_identity.yaml b/parm/jcb-gdas/model/atmosphere/background_error_static_identity.yaml similarity index 100% rename from parm/jcb-gdas/atmosphere/model/background_error_static_identity.yaml rename to parm/jcb-gdas/model/atmosphere/background_error_static_identity.yaml diff --git a/parm/jcb-gdas/atmosphere/model/final_increment_gaussian.yaml b/parm/jcb-gdas/model/atmosphere/final_increment_gaussian.yaml similarity index 100% rename from parm/jcb-gdas/atmosphere/model/final_increment_gaussian.yaml rename to parm/jcb-gdas/model/atmosphere/final_increment_gaussian.yaml diff --git a/parm/jcb-gdas/atmosphere/model/geometry_background.yaml b/parm/jcb-gdas/model/atmosphere/geometry_background.yaml similarity index 100% rename from parm/jcb-gdas/atmosphere/model/geometry_background.yaml rename to parm/jcb-gdas/model/atmosphere/geometry_background.yaml diff --git a/parm/jcb-gdas/atmosphere/model/model_pseudo.yaml b/parm/jcb-gdas/model/atmosphere/model_pseudo.yaml similarity index 100% rename from parm/jcb-gdas/atmosphere/model/model_pseudo.yaml rename to parm/jcb-gdas/model/atmosphere/model_pseudo.yaml diff --git a/parm/jcb-gdas/atmosphere/model/output_ensemble_increments_gaussian.yaml b/parm/jcb-gdas/model/atmosphere/output_ensemble_increments_gaussian.yaml similarity index 100% rename from parm/jcb-gdas/atmosphere/model/output_ensemble_increments_gaussian.yaml rename to parm/jcb-gdas/model/atmosphere/output_ensemble_increments_gaussian.yaml diff --git a/parm/jcb-gdas/atmosphere/model/posterior_output_gaussian.yaml b/parm/jcb-gdas/model/atmosphere/posterior_output_gaussian.yaml similarity index 100% rename from parm/jcb-gdas/atmosphere/model/posterior_output_gaussian.yaml rename to parm/jcb-gdas/model/atmosphere/posterior_output_gaussian.yaml diff --git a/parm/jcb-gdas/marine/model/geometry.yaml b/parm/jcb-gdas/model/marine/geometry.yaml similarity index 100% rename from parm/jcb-gdas/marine/model/geometry.yaml rename to parm/jcb-gdas/model/marine/geometry.yaml diff --git a/parm/jcb-gdas/snow/model/geometry.yaml b/parm/jcb-gdas/model/snow/geometry.yaml similarity index 100% rename from parm/jcb-gdas/snow/model/geometry.yaml rename to parm/jcb-gdas/model/snow/geometry.yaml diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml b/parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml new file mode 120000 index 000000000..8c289e1f2 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml @@ -0,0 +1 @@ +../atmosphere/aircraft.yaml \ No newline at end of file diff --git a/parm/jcb-gdas/atmosphere/observations/lgetkf_amsua_n19.yaml b/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml similarity index 100% rename from parm/jcb-gdas/atmosphere/observations/lgetkf_amsua_n19.yaml rename to parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml diff --git a/parm/jcb-gdas/atmosphere/observations/lgetkf_conventional_ps.yaml b/parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml similarity index 100% rename from parm/jcb-gdas/atmosphere/observations/lgetkf_conventional_ps.yaml rename to parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml diff --git a/parm/jcb-gdas/atmosphere/observations/aircraft.yaml b/parm/jcb-gdas/observations/atmosphere/aircraft.yaml similarity index 100% rename from parm/jcb-gdas/atmosphere/observations/aircraft.yaml rename to parm/jcb-gdas/observations/atmosphere/aircraft.yaml diff --git a/parm/jcb-gdas/atmosphere/observations/amsua_n19.yaml b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml similarity index 100% rename from parm/jcb-gdas/atmosphere/observations/amsua_n19.yaml rename to parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml From 0060c77cc738083654af5fcd1acd8f55a2b66372 Mon Sep 17 00:00:00 2001 From: danholdaway Date: Thu, 14 Mar 2024 11:17:21 -0400 Subject: [PATCH 010/199] rename to j2 --- .../aero/{geometry.yaml => geometry.yaml.j2} | 0 ...loop_1.yaml => 3dvar_outer_loop_1.yaml.j2} | 0 ...loop_2.yaml => 3dvar_outer_loop_2.yaml.j2} | 0 .../{background.yaml => background.yaml.j2} | 0 ...emble.yaml => background_ensemble.yaml.j2} | 0 ...background_error_hybrid_bump_bump.yaml.j2} | 0 ...ckground_error_hybrid_gsibec_bump.yaml.j2} | 0 ...l => background_error_static_bump.yaml.j2} | 0 ...=> background_error_static_gsibec.yaml.j2} | 0 ... background_error_static_identity.yaml.j2} | 0 ....yaml => final_increment_gaussian.yaml.j2} | 0 ...round.yaml => geometry_background.yaml.j2} | 0 ...model_pseudo.yaml => model_pseudo.yaml.j2} | 0 ...tput_ensemble_increments_gaussian.yaml.j2} | 0 ...yaml => posterior_output_gaussian.yaml.j2} | 0 .../{geometry.yaml => geometry.yaml.j2} | 0 .../snow/{geometry.yaml => geometry.yaml.j2} | 0 .../{aircraft.yaml => aircraft.yaml.j2} | 0 .../{amsua_n19.yaml => amsua_n19.yaml.j2} | 24 +++++++++---------- ...tional_ps.yaml => conventional_ps.yaml.j2} | 6 ++--- .../{aircraft.yaml => aircraft.yaml.j2} | 13 ++++++++++ .../{amsua_n19.yaml => amsua_n19.yaml.j2} | 0 22 files changed, 28 insertions(+), 15 deletions(-) rename parm/jcb-gdas/model/aero/{geometry.yaml => geometry.yaml.j2} (100%) rename parm/jcb-gdas/model/atmosphere/{3dvar_outer_loop_1.yaml => 3dvar_outer_loop_1.yaml.j2} (100%) rename parm/jcb-gdas/model/atmosphere/{3dvar_outer_loop_2.yaml => 3dvar_outer_loop_2.yaml.j2} (100%) rename parm/jcb-gdas/model/atmosphere/{background.yaml => background.yaml.j2} (100%) rename parm/jcb-gdas/model/atmosphere/{background_ensemble.yaml => background_ensemble.yaml.j2} (100%) rename parm/jcb-gdas/model/atmosphere/{background_error_hybrid_bump_bump.yaml => background_error_hybrid_bump_bump.yaml.j2} (100%) rename parm/jcb-gdas/model/atmosphere/{background_error_hybrid_gsibec_bump.yaml => background_error_hybrid_gsibec_bump.yaml.j2} (100%) rename parm/jcb-gdas/model/atmosphere/{background_error_static_bump.yaml => background_error_static_bump.yaml.j2} (100%) rename parm/jcb-gdas/model/atmosphere/{background_error_static_gsibec.yaml => background_error_static_gsibec.yaml.j2} (100%) rename parm/jcb-gdas/model/atmosphere/{background_error_static_identity.yaml => background_error_static_identity.yaml.j2} (100%) rename parm/jcb-gdas/model/atmosphere/{final_increment_gaussian.yaml => final_increment_gaussian.yaml.j2} (100%) rename parm/jcb-gdas/model/atmosphere/{geometry_background.yaml => geometry_background.yaml.j2} (100%) rename parm/jcb-gdas/model/atmosphere/{model_pseudo.yaml => model_pseudo.yaml.j2} (100%) rename parm/jcb-gdas/model/atmosphere/{output_ensemble_increments_gaussian.yaml => output_ensemble_increments_gaussian.yaml.j2} (100%) rename parm/jcb-gdas/model/atmosphere/{posterior_output_gaussian.yaml => posterior_output_gaussian.yaml.j2} (100%) rename parm/jcb-gdas/model/marine/{geometry.yaml => geometry.yaml.j2} (100%) rename parm/jcb-gdas/model/snow/{geometry.yaml => geometry.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere-lgetkf/{aircraft.yaml => aircraft.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere-lgetkf/{amsua_n19.yaml => amsua_n19.yaml.j2} (69%) rename parm/jcb-gdas/observations/atmosphere-lgetkf/{conventional_ps.yaml => conventional_ps.yaml.j2} (97%) rename parm/jcb-gdas/observations/atmosphere/{aircraft.yaml => aircraft.yaml.j2} (97%) rename parm/jcb-gdas/observations/atmosphere/{amsua_n19.yaml => amsua_n19.yaml.j2} (100%) diff --git a/parm/jcb-gdas/model/aero/geometry.yaml b/parm/jcb-gdas/model/aero/geometry.yaml.j2 similarity index 100% rename from parm/jcb-gdas/model/aero/geometry.yaml rename to parm/jcb-gdas/model/aero/geometry.yaml.j2 diff --git a/parm/jcb-gdas/model/atmosphere/3dvar_outer_loop_1.yaml b/parm/jcb-gdas/model/atmosphere/3dvar_outer_loop_1.yaml.j2 similarity index 100% rename from parm/jcb-gdas/model/atmosphere/3dvar_outer_loop_1.yaml rename to parm/jcb-gdas/model/atmosphere/3dvar_outer_loop_1.yaml.j2 diff --git a/parm/jcb-gdas/model/atmosphere/3dvar_outer_loop_2.yaml b/parm/jcb-gdas/model/atmosphere/3dvar_outer_loop_2.yaml.j2 similarity index 100% rename from parm/jcb-gdas/model/atmosphere/3dvar_outer_loop_2.yaml rename to parm/jcb-gdas/model/atmosphere/3dvar_outer_loop_2.yaml.j2 diff --git a/parm/jcb-gdas/model/atmosphere/background.yaml b/parm/jcb-gdas/model/atmosphere/background.yaml.j2 similarity index 100% rename from parm/jcb-gdas/model/atmosphere/background.yaml rename to parm/jcb-gdas/model/atmosphere/background.yaml.j2 diff --git a/parm/jcb-gdas/model/atmosphere/background_ensemble.yaml b/parm/jcb-gdas/model/atmosphere/background_ensemble.yaml.j2 similarity index 100% rename from parm/jcb-gdas/model/atmosphere/background_ensemble.yaml rename to parm/jcb-gdas/model/atmosphere/background_ensemble.yaml.j2 diff --git a/parm/jcb-gdas/model/atmosphere/background_error_hybrid_bump_bump.yaml b/parm/jcb-gdas/model/atmosphere/background_error_hybrid_bump_bump.yaml.j2 similarity index 100% rename from parm/jcb-gdas/model/atmosphere/background_error_hybrid_bump_bump.yaml rename to parm/jcb-gdas/model/atmosphere/background_error_hybrid_bump_bump.yaml.j2 diff --git a/parm/jcb-gdas/model/atmosphere/background_error_hybrid_gsibec_bump.yaml b/parm/jcb-gdas/model/atmosphere/background_error_hybrid_gsibec_bump.yaml.j2 similarity index 100% rename from parm/jcb-gdas/model/atmosphere/background_error_hybrid_gsibec_bump.yaml rename to parm/jcb-gdas/model/atmosphere/background_error_hybrid_gsibec_bump.yaml.j2 diff --git a/parm/jcb-gdas/model/atmosphere/background_error_static_bump.yaml b/parm/jcb-gdas/model/atmosphere/background_error_static_bump.yaml.j2 similarity index 100% rename from parm/jcb-gdas/model/atmosphere/background_error_static_bump.yaml rename to parm/jcb-gdas/model/atmosphere/background_error_static_bump.yaml.j2 diff --git a/parm/jcb-gdas/model/atmosphere/background_error_static_gsibec.yaml b/parm/jcb-gdas/model/atmosphere/background_error_static_gsibec.yaml.j2 similarity index 100% rename from parm/jcb-gdas/model/atmosphere/background_error_static_gsibec.yaml rename to parm/jcb-gdas/model/atmosphere/background_error_static_gsibec.yaml.j2 diff --git a/parm/jcb-gdas/model/atmosphere/background_error_static_identity.yaml b/parm/jcb-gdas/model/atmosphere/background_error_static_identity.yaml.j2 similarity index 100% rename from parm/jcb-gdas/model/atmosphere/background_error_static_identity.yaml rename to parm/jcb-gdas/model/atmosphere/background_error_static_identity.yaml.j2 diff --git a/parm/jcb-gdas/model/atmosphere/final_increment_gaussian.yaml b/parm/jcb-gdas/model/atmosphere/final_increment_gaussian.yaml.j2 similarity index 100% rename from parm/jcb-gdas/model/atmosphere/final_increment_gaussian.yaml rename to parm/jcb-gdas/model/atmosphere/final_increment_gaussian.yaml.j2 diff --git a/parm/jcb-gdas/model/atmosphere/geometry_background.yaml b/parm/jcb-gdas/model/atmosphere/geometry_background.yaml.j2 similarity index 100% rename from parm/jcb-gdas/model/atmosphere/geometry_background.yaml rename to parm/jcb-gdas/model/atmosphere/geometry_background.yaml.j2 diff --git a/parm/jcb-gdas/model/atmosphere/model_pseudo.yaml b/parm/jcb-gdas/model/atmosphere/model_pseudo.yaml.j2 similarity index 100% rename from parm/jcb-gdas/model/atmosphere/model_pseudo.yaml rename to parm/jcb-gdas/model/atmosphere/model_pseudo.yaml.j2 diff --git a/parm/jcb-gdas/model/atmosphere/output_ensemble_increments_gaussian.yaml b/parm/jcb-gdas/model/atmosphere/output_ensemble_increments_gaussian.yaml.j2 similarity index 100% rename from parm/jcb-gdas/model/atmosphere/output_ensemble_increments_gaussian.yaml rename to parm/jcb-gdas/model/atmosphere/output_ensemble_increments_gaussian.yaml.j2 diff --git a/parm/jcb-gdas/model/atmosphere/posterior_output_gaussian.yaml b/parm/jcb-gdas/model/atmosphere/posterior_output_gaussian.yaml.j2 similarity index 100% rename from parm/jcb-gdas/model/atmosphere/posterior_output_gaussian.yaml rename to parm/jcb-gdas/model/atmosphere/posterior_output_gaussian.yaml.j2 diff --git a/parm/jcb-gdas/model/marine/geometry.yaml b/parm/jcb-gdas/model/marine/geometry.yaml.j2 similarity index 100% rename from parm/jcb-gdas/model/marine/geometry.yaml rename to parm/jcb-gdas/model/marine/geometry.yaml.j2 diff --git a/parm/jcb-gdas/model/snow/geometry.yaml b/parm/jcb-gdas/model/snow/geometry.yaml.j2 similarity index 100% rename from parm/jcb-gdas/model/snow/geometry.yaml rename to parm/jcb-gdas/model/snow/geometry.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml b/parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml rename to parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml b/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 similarity index 69% rename from parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml rename to parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 index fe4c603c4..ed38842c3 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 @@ -6,37 +6,37 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}amsua_n19{{atm_obsdatain_suffix}}" + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: './diags/diag_amsua_n19_{{ current_cycle | to_YMDH }}.nc' + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [brightnessTemperature] - channels: &amsua_n19_channels 4-6,9-14 + channels: &{{observation_from_jcb}}_channels 4-6,9-14 obs operator: name: CRTM Absorbers: [H2O,O3] Clouds: [Water, Ice] Cloud_Fraction: 1.0 obs options: - Sensor_ID: amsua_n19 + Sensor_ID: {{observation_from_jcb}} EndianType: little_endian CoefficientPath: ./crtm/ obs error: covariance model: diagonal obs bias: - input file: "{{atm_obsbiasin_path}}/{{atm_obsbiasin_prefix}}amsua_n19{{atm_obsbiasin_suffix}}" - output file: "{{atm_obsbiasout_path}}/{{atm_obsbiasout_prefix}}amsua_n19{{atm_obsbiasout_suffix}}" + input file: "{{atm_obsbiasin_path}}/{{atm_obsbiasin_prefix}}{{observation_from_jcb}}{{atm_obsbiasin_suffix}}" + output file: "{{atm_obsbiasout_path}}/{{atm_obsbiasout_prefix}}{{observation_from_jcb}}{{atm_obsbiasout_suffix}}" variational bc: predictors: - name: constant - name: lapse_rate order: 2 - tlapse: &amsua_n19_tlapse "{{atm_obsbiasin_path}}/{{atm_obstlapsein_prefix}}amsua_n19{{atm_obstlapsein_suffix}}" + tlapse: &{{observation_from_jcb}}_tlapse "{{atm_obsbiasin_path}}/{{atm_obstlapsein_prefix}}{{observation_from_jcb}}{{atm_obstlapsein_suffix}}" - name: lapse_rate - tlapse: *amsua_n19_tlapse + tlapse: *{{observation_from_jcb}}_tlapse - name: emissivity - name: scan_angle order: 4 @@ -51,16 +51,16 @@ step size: 1.0e-4 largest analysis variance: 10000.0 prior: - input file: "{{atm_obsbiasin_path}}/{{atm_obsbiascovin_prefix}}amsua_n19{{atm_obsbiascovin_suffix}}" + input file: "{{atm_obsbiasin_path}}/{{atm_obsbiascovin_prefix}}{{observation_from_jcb}}{{atm_obsbiascovin_suffix}}" inflation: ratio: 1.1 ratio for small dataset: 2.0 - output file: "{{atm_obsbiasout_path}}/{{atm_obsbiascovout_prefix}}amsua_n19{{atm_obsbiascovout_suffix}}" + output file: "{{atm_obsbiasout_path}}/{{atm_obsbiascovout_prefix}}{{observation_from_jcb}}{{atm_obsbiascovout_suffix}}" obs filters: - filter: Bounds Check filter variables: - name: brightnessTemperature - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels minvalue: 100.0 maxvalue: 500.0 action: @@ -69,7 +69,7 @@ - filter: Background Check filter variables: - name: brightnessTemperature - channels: *amsua_n19_channels + channels: *{{observation_from_jcb}}_channels threshold: 3.0 action: name: reject diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml b/parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml.j2 similarity index 97% rename from parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml rename to parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml.j2 index 170baaba6..640493a7c 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml.j2 @@ -1,16 +1,16 @@ - obs space: - name: surface_ps + name: Surface Pressure distribution: name: Halo halo size: 1250e3 obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}conventional_ps{{atm_obsdatain_suffix}}" + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: '{{ DATA }}/diags/diag_conventional_ps_prepbufr_{{ current_cycle | to_YMDH }}.nc' + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [stationPressure] diff --git a/parm/jcb-gdas/observations/atmosphere/aircraft.yaml b/parm/jcb-gdas/observations/atmosphere/aircraft.yaml.j2 similarity index 97% rename from parm/jcb-gdas/observations/atmosphere/aircraft.yaml rename to parm/jcb-gdas/observations/atmosphere/aircraft.yaml.j2 index cc957e662..210ca82ad 100644 --- a/parm/jcb-gdas/observations/atmosphere/aircraft.yaml +++ b/parm/jcb-gdas/observations/atmosphere/aircraft.yaml.j2 @@ -1,5 +1,10 @@ - obs space: name: Aircraft + {% if algorithm == "local_ensemble_da" %} + distribution: + name: Halo + halo size: 1250e3 + {% endif %} obsdatain: engine: type: H5File @@ -15,6 +20,14 @@ io pool: max pool size: 1 simulated variables: [windEastward, windNorthward, airTemperature, specificHumidity] + + {% if algorithm == "local_ensemble_da" %} + obs localizations: + - localization method: Horizontal Gaspari-Cohn + lengthscale: 1250e3 + max nobs: 10000] + {% endif %} + # obs operator: name: VertInterp diff --git a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml rename to parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 From 9bf50ef76f9f8ca3bb19da5a7d84742cfc316245 Mon Sep 17 00:00:00 2001 From: danholdaway Date: Thu, 14 Mar 2024 11:19:45 -0400 Subject: [PATCH 011/199] j2 --- parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml.j2 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml.j2 index 8c289e1f2..9685f0007 120000 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml.j2 @@ -1 +1 @@ -../atmosphere/aircraft.yaml \ No newline at end of file +../atmosphere/aircraft.yaml.j2 \ No newline at end of file From 768a4e50ddc0b4216022ea89094c9c36c89ae132 Mon Sep 17 00:00:00 2001 From: danholdaway Date: Wed, 20 Mar 2024 11:59:14 -0400 Subject: [PATCH 012/199] add observer chronicles --- .../atmosphere/amsua_n19.yaml | 33 +++++++ .../atmosphere-lgetkf/amsua_n19.yaml.j2 | 2 +- .../observations/atmosphere/amsua_n19.yaml.j2 | 88 +++++++++---------- 3 files changed, 76 insertions(+), 47 deletions(-) create mode 100644 parm/jcb-gdas/observation_chronicle/atmosphere/amsua_n19.yaml diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/amsua_n19.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/amsua_n19.yaml new file mode 100644 index 000000000..2535b7f23 --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/atmosphere/amsua_n19.yaml @@ -0,0 +1,33 @@ +# Mission time frame +# ------------------ +commissioned: '2009-04-14T00:00:00' + + +# Observer type +# ------------- +observer_type: satellite + + +# Initial channel usage +# --------------------- +simulated_channels: 1-15 +active_channels: 1-15 +observation_errors: [0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5] + + +# Chronicle of data use from this instrument +# ------------------------------------------ +chronicles: + +- action_date: 2009-04-14T00:00:00 + justification: 'Set active channels for JEDI' + action: + type: remove_active_channels + channels: 7-8,14 + +- action_date: 2009-04-16T00:00:00 + justification: 'Set active channels for JEDI' + action: + type: adjust_observation_errors + channels: [4,5,6] + adjusted_errors: [0.75, 0.75, 0.75] diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 index ed38842c3..944e3e956 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 @@ -14,7 +14,7 @@ io pool: max pool size: 1 simulated variables: [brightnessTemperature] - channels: &{{observation_from_jcb}}_channels 4-6,9-14 + channels: &{{observation_from_jcb}}_channels {{ get_satellite_simulated_channels(observation_from_jcb) }} obs operator: name: CRTM Absorbers: [H2O,O3] diff --git a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 index f7ab997e4..eda1951a8 100644 --- a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 @@ -11,7 +11,7 @@ io pool: max pool size: 1 simulated variables: [brightnessTemperature] - channels: &{{observation_from_jcb}}_channels 1-15 + channels: &available_channels {{ get_satellite_simulated_channels(observation_from_jcb) }} obs operator: name: CRTM Absorbers: [H2O,O3] @@ -55,14 +55,14 @@ - filter: BlackList filter variables: - name: brightnessTemperature - channels: *{{observation_from_jcb}}_channels + channels: *available_channels action: name: assign error error function: name: ObsFunction/ObsErrorModelRamp - channels: *{{observation_from_jcb}}_channels + channels: *available_channels options: - channels: *{{observation_from_jcb}}_channels + channels: *available_channels xvar: name: ObsFunction/CLWRetSymmetricMW options: @@ -113,12 +113,12 @@ - filter: Bounds Check filter variables: - name: brightnessTemperature - channels: *{{observation_from_jcb}}_channels + channels: *available_channels test variables: - name: ObsFunction/HydrometeorCheckAMSUA - channels: *{{observation_from_jcb}}_channels + channels: *available_channels options: - channels: *{{observation_from_jcb}}_channels + channels: *available_channels obserr_clearsky: [ 2.500, 2.200, 2.000, 0.550, 0.300, 0.230, 0.230, 0.250, 0.250, 0.350, 0.400, 0.550, 0.800, 3.000, 3.500] @@ -130,9 +130,9 @@ clwret_types: [ObsValue] obserr_function: name: ObsFunction/ObsErrorModelRamp - channels: *{{observation_from_jcb}}_channels + channels: *available_channels options: - channels: *{{observation_from_jcb}}_channels + channels: *available_channels xvar: name: ObsFunction/CLWRetSymmetricMW options: @@ -158,56 +158,56 @@ - filter: BlackList filter variables: - name: brightnessTemperature - channels: *{{observation_from_jcb}}_channels + channels: *available_channels action: name: inflate error inflation variable: name: ObsFunction/ObsErrorFactorTopoRad - channels: *{{observation_from_jcb}}_channels + channels: *available_channels options: sensor: {{observation_from_jcb}} - channels: *{{observation_from_jcb}}_channels + channels: *available_channels # Transmittnace Top Check - filter: BlackList filter variables: - name: brightnessTemperature - channels: *{{observation_from_jcb}}_channels + channels: *available_channels action: name: inflate error inflation variable: name: ObsFunction/ObsErrorFactorTransmitTopRad - channels: *{{observation_from_jcb}}_channels + channels: *available_channels options: sensor: {{observation_from_jcb}} - channels: *{{observation_from_jcb}}_channels + channels: *available_channels # Surface Jacobian check - filter: BlackList filter variables: - name: brightnessTemperature - channels: *{{observation_from_jcb}}_channels + channels: *available_channels action: name: inflate error inflation variable: name: ObsFunction/ObsErrorFactorSurfJacobianRad - channels: *{{observation_from_jcb}}_channels + channels: *available_channels options: sensor: {{observation_from_jcb}} - channels: *{{observation_from_jcb}}_channels + channels: *available_channels obserr_demisf: [0.010, 0.020, 0.015, 0.020, 0.200] obserr_dtempf: [0.500, 2.000, 1.000, 2.000, 4.500] # Situation dependent Check - filter: BlackList filter variables: - name: brightnessTemperature - channels: *{{observation_from_jcb}}_channels + channels: *available_channels action: name: inflate error inflation variable: name: ObsFunction/ObsErrorFactorSituDependMW - channels: *{{observation_from_jcb}}_channels + channels: *available_channels options: sensor: {{observation_from_jcb}} - channels: *{{observation_from_jcb}}_channels + channels: *available_channels clwobs_function: name: ObsFunction/CLWRetMW options: @@ -231,9 +231,9 @@ bias_application: HofX clwmatchidx_function: name: ObsFunction/CLWMatchIndexMW - channels: *{{observation_from_jcb}}_channels + channels: *available_channels options: - channels: *{{observation_from_jcb}}_channels + channels: *available_channels clwobs_function: name: ObsFunction/CLWRetMW options: @@ -252,9 +252,9 @@ 0.000, 0.000, 0.000, 0.000, 0.030] obserr_function: name: ObsFunction/ObsErrorModelRamp - channels: *{{observation_from_jcb}}_channels + channels: *available_channels options: - channels: *{{observation_from_jcb}}_channels + channels: *available_channels xvar: name: ObsFunction/CLWRetSymmetricMW options: @@ -280,33 +280,33 @@ - filter: Background Check filter variables: - name: brightnessTemperature - channels: *{{observation_from_jcb}}_channels + channels: *available_channels function absolute threshold: - name: ObsFunction/ObsErrorBoundMW - channels: *{{observation_from_jcb}}_channels + channels: *available_channels options: sensor: {{observation_from_jcb}} - channels: *{{observation_from_jcb}}_channels + channels: *available_channels obserr_bound_latitude: name: ObsFunction/ObsErrorFactorLatRad options: latitude_parameters: [25.0, 0.25, 0.04, 3.0] obserr_bound_transmittop: name: ObsFunction/ObsErrorFactorTransmitTopRad - channels: *{{observation_from_jcb}}_channels + channels: *available_channels options: - channels: *{{observation_from_jcb}}_channels + channels: *available_channels obserr_bound_topo: name: ObsFunction/ObsErrorFactorTopoRad - channels: *{{observation_from_jcb}}_channels + channels: *available_channels options: - channels: *{{observation_from_jcb}}_channels + channels: *available_channels sensor: {{observation_from_jcb}} obserr_function: name: ObsFunction/ObsErrorModelRamp - channels: *{{observation_from_jcb}}_channels + channels: *available_channels options: - channels: *{{observation_from_jcb}}_channels + channels: *available_channels xvar: name: ObsFunction/CLWRetSymmetricMW options: @@ -335,16 +335,14 @@ - filter: Bounds Check filter variables: - name: brightnessTemperature - channels: *{{observation_from_jcb}}_channels + channels: *available_channels test variables: - name: ObsFunction/InterChannelConsistencyCheck - channels: *{{observation_from_jcb}}_channels + channels: *available_channels options: - channels: *{{observation_from_jcb}}_channels + channels: *available_channels sensor: {{observation_from_jcb}} - use_flag: [ 1, 1, 1, 1, 1, - 1, -1, -1, 1, 1, - 1, 1, 1, -1, 1 ] + use_flag: &active_channels {{ get_satellite_active_channels(observation_from_jcb) }} maxvalue: 1.0e-12 action: name: reject @@ -352,16 +350,14 @@ - filter: Bounds Check filter variables: - name: brightnessTemperature - channels: *{{observation_from_jcb}}_channels + channels: *available_channels test variables: - name: ObsFunction/ChannelUseflagCheckRad - channels: *{{observation_from_jcb}}_channels + channels: *available_channels options: sensor: {{observation_from_jcb}} - channels: *{{observation_from_jcb}}_channels - use_flag: [ 1, 1, 1, 1, 1, - 1, -1, -1, 1, 1, - 1, 1, 1, -1, 1 ] + channels: *available_channels + use_flag: *active_channels minvalue: 1.0e-12 action: name: reject From 5a71f1777ee1611babf55440d2b5920eaf2a7c8a Mon Sep 17 00:00:00 2001 From: danholdaway Date: Wed, 20 Mar 2024 16:33:53 -0400 Subject: [PATCH 013/199] update chron for amsua n19 --- .../atmosphere/amsua_n19.yaml | 30 +------------------ 1 file changed, 1 insertion(+), 29 deletions(-) diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/amsua_n19.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/amsua_n19.yaml index 2535b7f23..614c19188 100644 --- a/parm/jcb-gdas/observation_chronicle/atmosphere/amsua_n19.yaml +++ b/parm/jcb-gdas/observation_chronicle/atmosphere/amsua_n19.yaml @@ -1,33 +1,5 @@ -# Mission time frame -# ------------------ commissioned: '2009-04-14T00:00:00' - - -# Observer type -# ------------- observer_type: satellite - -# Initial channel usage -# --------------------- simulated_channels: 1-15 -active_channels: 1-15 -observation_errors: [0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5] - - -# Chronicle of data use from this instrument -# ------------------------------------------ -chronicles: - -- action_date: 2009-04-14T00:00:00 - justification: 'Set active channels for JEDI' - action: - type: remove_active_channels - channels: 7-8,14 - -- action_date: 2009-04-16T00:00:00 - justification: 'Set active channels for JEDI' - action: - type: adjust_observation_errors - channels: [4,5,6] - adjusted_errors: [0.75, 0.75, 0.75] +active_channels: 1-6,9-13,15 From 294b63079a54e60294362b21ed666dee8911ece5 Mon Sep 17 00:00:00 2001 From: danholdaway Date: Wed, 27 Mar 2024 13:18:38 -0400 Subject: [PATCH 014/199] put all amsua n19 variables into chronicle --- .../atmosphere/amsua_n19.yaml | 19 ++- .../observations/atmosphere/amsua_n19.yaml.j2 | 160 +++++++----------- 2 files changed, 78 insertions(+), 101 deletions(-) diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/amsua_n19.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/amsua_n19.yaml index 614c19188..e3293aeb4 100644 --- a/parm/jcb-gdas/observation_chronicle/atmosphere/amsua_n19.yaml +++ b/parm/jcb-gdas/observation_chronicle/atmosphere/amsua_n19.yaml @@ -1,5 +1,22 @@ commissioned: '2009-04-14T00:00:00' observer_type: satellite -simulated_channels: 1-15 +# Instrument initial configuration +simulated_channels: [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] +channel_dependent_variables: + x0: + values: [ 0.050, 0.030, 0.030, 0.020, 0.000, 0.100, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.030] + x1: + values: [ 0.600, 0.450, 0.400, 0.450, 1.000, 1.500, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.200] + err0: + values: [ 2.500, 2.200, 2.000, 0.550, 0.300, 0.230, 0.230, 0.250, 0.250, 0.350, 0.400, 0.550, 0.800, 3.000, 3.500] + err1: + values: [20.000, 18.000, 12.000, 3.000, 0.500, 0.300, 0.230, 0.250, 0.250, 0.350, 0.400, 0.550, 0.800, 3.000, 18.000] + obserr_clearsky: + values: [ 2.500, 2.200, 2.000, 0.550, 0.300, 0.230, 0.230, 0.250, 0.250, 0.350, 0.400, 0.550, 0.800, 3.000, 3.500] + clwret_clearsky: + values: [ 0.050, 0.030, 0.030, 0.020, 0.000, 0.100, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.030] + obserr_bound_max: + values: [ 4.5, 4.5, 4.5, 2.5, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.5, 3.5, 4.5, 4.5, 4.5] active_channels: 1-6,9-13,15 + diff --git a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 index eda1951a8..9ee108537 100644 --- a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 @@ -11,7 +11,7 @@ io pool: max pool size: 1 simulated variables: [brightnessTemperature] - channels: &available_channels {{ get_satellite_simulated_channels(observation_from_jcb) }} + channels: &simulated_channels {{ get_satellite_simulated_channels(observation_from_jcb) }} obs operator: name: CRTM Absorbers: [H2O,O3] @@ -55,32 +55,24 @@ - filter: BlackList filter variables: - name: brightnessTemperature - channels: *available_channels + channels: *simulated_channels action: name: assign error error function: name: ObsFunction/ObsErrorModelRamp - channels: *available_channels + channels: *simulated_channels options: - channels: *available_channels + channels: *simulated_channels xvar: name: ObsFunction/CLWRetSymmetricMW options: clwret_ch238: 1 clwret_ch314: 2 clwret_types: [ObsValue, HofX] - x0: [ 0.050, 0.030, 0.030, 0.020, 0.000, - 0.100, 0.000, 0.000, 0.000, 0.000, - 0.000, 0.000, 0.000, 0.000, 0.030] - x1: [ 0.600, 0.450, 0.400, 0.450, 1.000, - 1.500, 0.000, 0.000, 0.000, 0.000, - 0.000, 0.000, 0.000, 0.000, 0.200] - err0: [ 2.500, 2.200, 2.000, 0.550, 0.300, - 0.230, 0.230, 0.250, 0.250, 0.350, - 0.400, 0.550, 0.800, 3.000, 3.500] - err1: [20.000, 18.000, 12.000, 3.000, 0.500, - 0.300, 0.230, 0.250, 0.250, 0.350, - 0.400, 0.550, 0.800, 3.000, 18.000] + x0: &x0 {{ get_satellite_channel_dep_variable(observation_from_jcb, "x0") }} + x1: &x1 {{ get_satellite_channel_dep_variable(observation_from_jcb, "x1") }} + err0: &err0 {{ get_satellite_channel_dep_variable(observation_from_jcb, "err0") }} + err1: &err1 {{ get_satellite_channel_dep_variable(observation_from_jcb, "err1") }} # CLW Retrieval Check - filter: Bounds Check filter variables: @@ -113,15 +105,13 @@ - filter: Bounds Check filter variables: - name: brightnessTemperature - channels: *available_channels + channels: *simulated_channels test variables: - name: ObsFunction/HydrometeorCheckAMSUA - channels: *available_channels + channels: *simulated_channels options: - channels: *available_channels - obserr_clearsky: [ 2.500, 2.200, 2.000, 0.550, 0.300, - 0.230, 0.230, 0.250, 0.250, 0.350, - 0.400, 0.550, 0.800, 3.000, 3.500] + channels: *simulated_channels + obserr_clearsky: &obserr_clearsky {{ get_satellite_channel_dep_variable(observation_from_jcb, "obserr_clearsky") }} clwret_function: name: ObsFunction/CLWRetMW options: @@ -130,27 +120,19 @@ clwret_types: [ObsValue] obserr_function: name: ObsFunction/ObsErrorModelRamp - channels: *available_channels + channels: *simulated_channels options: - channels: *available_channels + channels: *simulated_channels xvar: name: ObsFunction/CLWRetSymmetricMW options: clwret_ch238: 1 clwret_ch314: 2 clwret_types: [ObsValue, HofX] - x0: [ 0.050, 0.030, 0.030, 0.020, 0.000, - 0.100, 0.000, 0.000, 0.000, 0.000, - 0.000, 0.000, 0.000, 0.000, 0.030] - x1: [ 0.600, 0.450, 0.400, 0.450, 1.000, - 1.500, 0.000, 0.000, 0.000, 0.000, - 0.000, 0.000, 0.000, 0.000, 0.200] - err0: [ 2.500, 2.200, 2.000, 0.550, 0.300, - 0.230, 0.230, 0.250, 0.250, 0.350, - 0.400, 0.550, 0.800, 3.000, 3.500] - err1: [20.000, 18.000, 12.000, 3.000, 0.500, - 0.300, 0.230, 0.250, 0.250, 0.350, - 0.400, 0.550, 0.800, 3.000, 18.000] + x0: *x0 + x1: *x1 + err0: *err0 + err1: *err1 maxvalue: 0.0 action: name: reject @@ -158,56 +140,56 @@ - filter: BlackList filter variables: - name: brightnessTemperature - channels: *available_channels + channels: *simulated_channels action: name: inflate error inflation variable: name: ObsFunction/ObsErrorFactorTopoRad - channels: *available_channels + channels: *simulated_channels options: sensor: {{observation_from_jcb}} - channels: *available_channels + channels: *simulated_channels # Transmittnace Top Check - filter: BlackList filter variables: - name: brightnessTemperature - channels: *available_channels + channels: *simulated_channels action: name: inflate error inflation variable: name: ObsFunction/ObsErrorFactorTransmitTopRad - channels: *available_channels + channels: *simulated_channels options: sensor: {{observation_from_jcb}} - channels: *available_channels + channels: *simulated_channels # Surface Jacobian check - filter: BlackList filter variables: - name: brightnessTemperature - channels: *available_channels + channels: *simulated_channels action: name: inflate error inflation variable: name: ObsFunction/ObsErrorFactorSurfJacobianRad - channels: *available_channels + channels: *simulated_channels options: sensor: {{observation_from_jcb}} - channels: *available_channels + channels: *simulated_channels obserr_demisf: [0.010, 0.020, 0.015, 0.020, 0.200] obserr_dtempf: [0.500, 2.000, 1.000, 2.000, 4.500] # Situation dependent Check - filter: BlackList filter variables: - name: brightnessTemperature - channels: *available_channels + channels: *simulated_channels action: name: inflate error inflation variable: name: ObsFunction/ObsErrorFactorSituDependMW - channels: *available_channels + channels: *simulated_channels options: sensor: {{observation_from_jcb}} - channels: *available_channels + channels: *simulated_channels clwobs_function: name: ObsFunction/CLWRetMW options: @@ -231,9 +213,9 @@ bias_application: HofX clwmatchidx_function: name: ObsFunction/CLWMatchIndexMW - channels: *available_channels + channels: *simulated_channels options: - channels: *available_channels + channels: *simulated_channels clwobs_function: name: ObsFunction/CLWRetMW options: @@ -247,66 +229,54 @@ clwret_ch314: 2 clwret_types: [HofX] bias_application: HofX - clwret_clearsky: [0.050, 0.030, 0.030, 0.020, 0.000, - 0.100, 0.000, 0.000, 0.000, 0.000, - 0.000, 0.000, 0.000, 0.000, 0.030] + clwret_clearsky: {{ get_satellite_channel_dep_variable(observation_from_jcb, "clwret_clearsky") }} obserr_function: name: ObsFunction/ObsErrorModelRamp - channels: *available_channels + channels: *simulated_channels options: - channels: *available_channels + channels: *simulated_channels xvar: name: ObsFunction/CLWRetSymmetricMW options: clwret_ch238: 1 clwret_ch314: 2 clwret_types: [ObsValue, HofX] - x0: [ 0.050, 0.030, 0.030, 0.020, 0.000, - 0.100, 0.000, 0.000, 0.000, 0.000, - 0.000, 0.000, 0.000, 0.000, 0.030] - x1: [ 0.600, 0.450, 0.400, 0.450, 1.000, - 1.500, 0.000, 0.000, 0.000, 0.000, - 0.000, 0.000, 0.000, 0.000, 0.200] - err0: [ 2.500, 2.200, 2.000, 0.550, 0.300, - 0.230, 0.230, 0.250, 0.250, 0.350, - 0.400, 0.550, 0.800, 3.000, 3.500] - err1: [20.000, 18.000, 12.000, 3.000, 0.500, - 0.300, 0.230, 0.250, 0.250, 0.350, - 0.400, 0.550, 0.800, 3.000, 18.000] - obserr_clearsky: [2.500, 2.200, 2.000, 0.550, 0.300, - 0.230, 0.230, 0.250, 0.250, 0.350, - 0.400, 0.550, 0.800, 3.000, 3.500] + x0: *x0 + x1: *x1 + err0: *err0 + err1: *err1 + obserr_clearsky: *obserr_clearsky # Gross check - filter: Background Check filter variables: - name: brightnessTemperature - channels: *available_channels + channels: *simulated_channels function absolute threshold: - name: ObsFunction/ObsErrorBoundMW - channels: *available_channels + channels: *simulated_channels options: sensor: {{observation_from_jcb}} - channels: *available_channels + channels: *simulated_channels obserr_bound_latitude: name: ObsFunction/ObsErrorFactorLatRad options: latitude_parameters: [25.0, 0.25, 0.04, 3.0] obserr_bound_transmittop: name: ObsFunction/ObsErrorFactorTransmitTopRad - channels: *available_channels + channels: *simulated_channels options: - channels: *available_channels + channels: *simulated_channels obserr_bound_topo: name: ObsFunction/ObsErrorFactorTopoRad - channels: *available_channels + channels: *simulated_channels options: - channels: *available_channels + channels: *simulated_channels sensor: {{observation_from_jcb}} obserr_function: name: ObsFunction/ObsErrorModelRamp - channels: *available_channels + channels: *simulated_channels options: - channels: *available_channels + channels: *simulated_channels xvar: name: ObsFunction/CLWRetSymmetricMW options: @@ -314,33 +284,23 @@ clwret_ch314: 2 clwret_types: [ObsValue, HofX] bias_application: HofX - x0: [ 0.050, 0.030, 0.030, 0.020, 0.000, - 0.100, 0.000, 0.000, 0.000, 0.000, - 0.000, 0.000, 0.000, 0.000, 0.030] - x1: [ 0.600, 0.450, 0.400, 0.450, 1.000, - 1.500, 0.000, 0.000, 0.000, 0.000, - 0.000, 0.000, 0.000, 0.000, 0.200] - err0: [ 2.500, 2.200, 2.000, 0.550, 0.300, - 0.230, 0.230, 0.250, 0.250, 0.350, - 0.400, 0.550, 0.800, 3.000, 3.500] - err1: [20.000, 18.000, 12.000, 3.000, 0.500, - 0.300, 0.230, 0.250, 0.250, 0.350, - 0.400, 0.550, 0.800, 3.000, 18.000] - obserr_bound_max: [4.5, 4.5, 4.5, 2.5, 2.0, - 2.0, 2.0, 2.0, 2.0, 2.0, - 2.5, 3.5, 4.5, 4.5, 4.5] + x0: *x0 + x1: *x1 + err0: *err0 + err1: *err1 + obserr_bound_max: {{ get_satellite_channel_dep_variable(observation_from_jcb, "obserr_bound_max") }} action: name: reject # Inter-channel check - filter: Bounds Check filter variables: - name: brightnessTemperature - channels: *available_channels + channels: *simulated_channels test variables: - name: ObsFunction/InterChannelConsistencyCheck - channels: *available_channels + channels: *simulated_channels options: - channels: *available_channels + channels: *simulated_channels sensor: {{observation_from_jcb}} use_flag: &active_channels {{ get_satellite_active_channels(observation_from_jcb) }} maxvalue: 1.0e-12 @@ -350,13 +310,13 @@ - filter: Bounds Check filter variables: - name: brightnessTemperature - channels: *available_channels + channels: *simulated_channels test variables: - name: ObsFunction/ChannelUseflagCheckRad - channels: *available_channels + channels: *simulated_channels options: sensor: {{observation_from_jcb}} - channels: *available_channels + channels: *simulated_channels use_flag: *active_channels minvalue: 1.0e-12 action: From 3f280f0b70f453c3c7d049beea35538a2f1f527e Mon Sep 17 00:00:00 2001 From: danholdaway Date: Tue, 2 Apr 2024 15:55:48 -0400 Subject: [PATCH 015/199] Add obs YAMLS from gdas and create associated chronicles --- .../atmosphere/amsua_n19.yaml | 66 +- .../atmosphere/atms_n20.yaml | 43 + .../atmosphere/atms_npp.yaml | 43 + .../atmosphere/cris-fsr_n20.yaml | 452 ++++++++ .../atmosphere/cris-fsr_npp.yaml | 452 ++++++++ .../atmosphere/iasi_metop-a.yaml | 638 ++++++++++++ .../atmosphere/iasi_metop-b.yaml | 637 ++++++++++++ .../atmosphere/ssmis_f17.yaml | 45 + .../atmosphere/ssmis_f18.yaml | 45 + .../atmosphere-lgetkf/aircraft.yaml.j2 | 1 - .../atmosphere-lgetkf/amsua_n19.yaml.j2 | 45 +- .../atmosphere-lgetkf/conv_ps.yaml.j2 | 373 +++++++ .../atmosphere-lgetkf/gnssro.yaml.j2 | 167 +++ .../satwind_abi_goes-16.yaml.j2 | 611 +++++++++++ .../atmosphere-lgetkf/satwind_ahi_h8.yaml.j2 | 442 ++++++++ .../scatwind_ascat_metop-b.yaml.j2 | 323 ++++++ .../atmosphere-lgetkf/sondes.yaml.j2 | 59 ++ .../observations/atmosphere/aircraft.yaml.j2 | 475 ++++----- .../observations/atmosphere/amsua_n19.yaml.j2 | 191 ++-- .../observations/atmosphere/atms_n20.yaml.j2 | 537 ++++++++++ .../observations/atmosphere/atms_npp.yaml.j2 | 555 ++++++++++ .../conv_ps.yaml.j2} | 78 +- .../atmosphere/cris-fsr_n20.yaml.j2 | 446 ++++++++ .../atmosphere/cris-fsr_npp.yaml.j2 | 445 ++++++++ .../observations/atmosphere/gnssro.yaml.j2 | 157 +++ .../atmosphere/iasi_metop-a.yaml.j2 | 579 +++++++++++ .../atmosphere/iasi_metop-b.yaml.j2 | 579 +++++++++++ .../observations/atmosphere/omi_aura.yaml.j2 | 175 ++++ .../atmosphere/ompsnp_npp.yaml.j2 | 330 ++++++ .../atmosphere/ompstc_npp.yaml.j2 | 135 +++ .../observations/atmosphere/satwind.yaml.j2 | 982 ++++++++++++++++++ .../atmosphere/satwind_abi_goes-16.yaml.j2 | 601 +++++++++++ .../atmosphere/satwind_abi_goes-17.yaml.j2 | 601 +++++++++++ .../atmosphere/satwind_ahi_h8.yaml.j2 | 432 ++++++++ .../atmosphere/satwind_seviri_m11.yaml.j2 | 439 ++++++++ .../atmosphere/satwind_seviri_m8.yaml.j2 | 439 ++++++++ .../atmosphere/scatwind_ascat_metop-a.yaml.j2 | 313 ++++++ .../atmosphere/scatwind_ascat_metop-b.yaml.j2 | 313 ++++++ .../observations/atmosphere/sfc.yaml.j2 | 40 + .../observations/atmosphere/sfcship.yaml.j2 | 100 ++ .../observations/atmosphere/sondes.yaml.j2 | 128 +++ .../observations/atmosphere/ssmis_f17.yaml.j2 | 177 ++++ .../observations/atmosphere/ssmis_f18.yaml.j2 | 177 ++++ 43 files changed, 13489 insertions(+), 377 deletions(-) create mode 100644 parm/jcb-gdas/observation_chronicle/atmosphere/atms_n20.yaml create mode 100644 parm/jcb-gdas/observation_chronicle/atmosphere/atms_npp.yaml create mode 100644 parm/jcb-gdas/observation_chronicle/atmosphere/cris-fsr_n20.yaml create mode 100644 parm/jcb-gdas/observation_chronicle/atmosphere/cris-fsr_npp.yaml create mode 100644 parm/jcb-gdas/observation_chronicle/atmosphere/iasi_metop-a.yaml create mode 100644 parm/jcb-gdas/observation_chronicle/atmosphere/iasi_metop-b.yaml create mode 100644 parm/jcb-gdas/observation_chronicle/atmosphere/ssmis_f17.yaml create mode 100644 parm/jcb-gdas/observation_chronicle/atmosphere/ssmis_f18.yaml delete mode 120000 parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere-lgetkf/conv_ps.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere-lgetkf/gnssro.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere-lgetkf/satwind_abi_goes-16.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere-lgetkf/satwind_ahi_h8.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere-lgetkf/scatwind_ascat_metop-b.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere-lgetkf/sondes.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 rename parm/jcb-gdas/observations/{atmosphere-lgetkf/conventional_ps.yaml.j2 => atmosphere/conv_ps.yaml.j2} (84%) create mode 100644 parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/cris-fsr_npp.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/gnssro.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/omi_aura.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/ompsnp_npp.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/ompstc_npp.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/satwind.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/satwind_abi_goes-16.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/satwind_abi_goes-17.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/satwind_ahi_h8.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/satwind_seviri_m11.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/satwind_seviri_m8.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/scatwind_ascat_metop-a.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/scatwind_ascat_metop-b.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/sfc.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/sfcship.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/amsua_n19.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/amsua_n19.yaml index e3293aeb4..5a8e3ab4a 100644 --- a/parm/jcb-gdas/observation_chronicle/atmosphere/amsua_n19.yaml +++ b/parm/jcb-gdas/observation_chronicle/atmosphere/amsua_n19.yaml @@ -1,22 +1,50 @@ -commissioned: '2009-04-14T00:00:00' -observer_type: satellite +# Instrument metadata +# ------------------- +commissioned: 2009-04-14T00:00:00 + +observer_type: satellite # Type of chronicle to use # Instrument initial configuration -simulated_channels: [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] -channel_dependent_variables: - x0: - values: [ 0.050, 0.030, 0.030, 0.020, 0.000, 0.100, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.030] - x1: - values: [ 0.600, 0.450, 0.400, 0.450, 1.000, 1.500, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.200] - err0: - values: [ 2.500, 2.200, 2.000, 0.550, 0.300, 0.230, 0.230, 0.250, 0.250, 0.350, 0.400, 0.550, 0.800, 3.000, 3.500] - err1: - values: [20.000, 18.000, 12.000, 3.000, 0.500, 0.300, 0.230, 0.250, 0.250, 0.350, 0.400, 0.550, 0.800, 3.000, 18.000] - obserr_clearsky: - values: [ 2.500, 2.200, 2.000, 0.550, 0.300, 0.230, 0.230, 0.250, 0.250, 0.350, 0.400, 0.550, 0.800, 3.000, 3.500] - clwret_clearsky: - values: [ 0.050, 0.030, 0.030, 0.020, 0.000, 0.100, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.030] - obserr_bound_max: - values: [ 4.5, 4.5, 4.5, 2.5, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.5, 3.5, 4.5, 4.5, 4.5] -active_channels: 1-6,9-13,15 +# -------------------------------- +channel_variables: + simulated: min + active: min + biascorrtd: min + error: max + error_cld: max + ermax: max + var_b: min + var_pg: min + icld_det: min + icloud: min + iaerosol: min +channel_values: + 1: [ 1, 1, 1, 2.50, 20.00, 4.50, 10.00, 0.00, -2, 1, -1 ] + 2: [ 1, 1, 1, 2.20, 18.00, 4.50, 10.00, 0.00, -2, 1, -1 ] + 3: [ 1, 1, 1, 2.00, 12.00, 4.50, 10.00, 0.00, -2, 1, -1 ] + 4: [ 1, 1, 1, 0.55, 3.00, 2.50, 10.00, 0.00, -2, 1, -1 ] + 5: [ 1, 1, 1, 0.30, 0.50, 2.00, 10.00, 0.00, -2, 1, -1 ] + 6: [ 1, 1, 1, 0.23, 0.30, 2.00, 10.00, 0.00, -2, 1, -1 ] + 7: [ 1, 1, 1, 0.23, 0.23, 2.00, 10.00, 0.00, -2, 1, -1 ] + 8: [ 1, 1, 1, 0.25, 0.25, 2.00, 10.00, 0.00, -2, 1, -1 ] + 9: [ 1, 1, 1, 0.25, 0.25, 2.00, 10.00, 0.00, -2, 1, -1 ] + 10: [ 1, 1, 1, 0.35, 0.35, 2.00, 10.00, 0.00, -2, 1, -1 ] + 11: [ 1, 1, 1, 0.40, 0.40, 2.50, 10.00, 0.00, -2, 1, -1 ] + 12: [ 1, 1, 1, 0.55, 0.55, 3.50, 10.00, 0.00, -2, 1, -1 ] + 13: [ 1, 1, 1, 0.80, 0.80, 4.50, 10.00, 0.00, -2, 1, -1 ] + 14: [ 1, 1, 0, 4.00, 4.00, 4.50, 10.00, 0.00, -2, 1, -1 ] + 15: [ 1, 1, 1, 3.50, 18.00, 4.50, 10.00, 0.00, -2, 1, -1 ] + +# Chronicle of changes for this instrument +# ---------------------------------------- +chronicle: + +- action_date: "2009-12-22T00:00:00" + justification: 'Ch8 noisy (noise started on 12/21/2009). Switch to monitoring only' + channel_values: + 8: [ 1, -1, 1, 0.25, 0.25, 2.00, 10.00, 0.00, -2, 1, -1 ] +- action_date: "2009-12-22T00:00:00" + justification: 'Per NCEP r35918. Ch 7 switch to monitoring only' + channel_values: + 7: [ 1, -1, 1, 0.23, 0.23, 2.00, 10.00, 0.00, -2, 1, -1 ] diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/atms_n20.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/atms_n20.yaml new file mode 100644 index 000000000..40f02a5d6 --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/atmosphere/atms_n20.yaml @@ -0,0 +1,43 @@ +# Instrument metadata +# ------------------- +commissioned: 2018-05-10T12:00:00 + +observer_type: satellite # Type of chronicle to use + +# Instrument initial configuration +# -------------------------------- +channel_variables: + simulated: min + active: min + biascorrtd: min + error: max + error_cld: max + ermax: max + var_b: min + var_pg: min + icld_det: min + icloud: min + iaerosol: min +channel_values: + 1: [ 1, 1, 1, 4.50, 20.00, 4.50, 10.00, 0.00, -2, 1, -1 ] + 2: [ 1, 1, 1, 4.50, 25.00, 4.50, 10.00, 0.00, -2, 1, -1 ] + 3: [ 1, 1, 1, 4.50, 12.00, 3.00, 10.00, 0.00, -2, 1, -1 ] + 4: [ 1, 1, 1, 2.50, 7.00, 3.00, 10.00, 0.00, -2, 1, -1 ] + 5: [ 1, 1, 1, 0.55, 3.50, 1.00, 10.00, 0.00, -2, 1, -1 ] + 6: [ 1, 1, 1, 0.30, 3.00, 1.00, 10.00, 0.00, -2, 1, -1 ] + 7: [ 1, 1, 1, 0.30, 0.80, 1.00, 10.00, 0.00, -2, 1, -1 ] + 8: [ 1, 1, 1, 0.40, 0.40, 1.00, 10.00, 0.00, -2, 1, -1 ] + 9: [ 1, 1, 1, 0.40, 0.40, 1.00, 10.00, 0.00, -2, 1, -1 ] + 10: [ 1, 1, 1, 0.40, 0.40, 1.00, 10.00, 0.00, -2, 1, -1 ] + 11: [ 1, 1, 1, 0.45, 0.45, 1.00, 10.00, 0.00, -2, 1, -1 ] + 12: [ 1, 1, 1, 0.45, 0.45, 1.00, 10.00, 0.00, -2, 1, -1 ] + 13: [ 1, 1, 1, 0.55, 0.55, 1.00, 10.00, 0.00, -2, 1, -1 ] + 14: [ 1, 1, 1, 0.80, 0.80, 2.00, 10.00, 0.00, -2, 1, -1 ] + 15: [ 1, 1, 0, 4.00, 4.00, 4.50, 10.00, 0.00, -2, 1, -1 ] + 16: [ 1, 1, 1, 4.00, 19.00, 4.50, 10.00, 0.00, -2, 1, -1 ] + 17: [ 1, 1, 1, 4.00, 30.00, 2.00, 10.00, 0.00, -2, 1, -1 ] + 18: [ 1, 1, 1, 3.50, 25.00, 2.00, 10.00, 0.00, -2, 1, -1 ] + 19: [ 1, 1, 1, 3.00, 16.50, 2.00, 10.00, 0.00, -2, 1, -1 ] + 20: [ 1, 1, 1, 3.00, 12.00, 2.00, 10.00, 0.00, -2, 1, -1 ] + 21: [ 1, 1, 1, 3.00, 9.00, 2.00, 10.00, 0.00, -2, 1, -1 ] + 22: [ 1, 1, 1, 3.00, 6.50, 2.00, 10.00, 0.00, -2, 1, -1 ] diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/atms_npp.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/atms_npp.yaml new file mode 100644 index 000000000..6a83f603b --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/atmosphere/atms_npp.yaml @@ -0,0 +1,43 @@ +# Instrument metadata +# ------------------- +commissioned: 2011-11-16T00:00:00 + +observer_type: satellite # Type of chronicle to use + +# Instrument initial configuration +# -------------------------------- +channel_variables: + simulated: min + active: min + biascorrtd: min + error: max + error_cld: max + ermax: max + var_b: min + var_pg: min + icld_det: min + icloud: min + iaerosol: min +channel_values: + 1: [ 1, 1, 1, 4.50, 20.00, 4.50, 10.00, 0.00, -2, 1, -1 ] + 2: [ 1, 1, 1, 4.50, 25.00, 4.50, 10.00, 0.00, -2, 1, -1 ] + 3: [ 1, 1, 1, 4.50, 12.00, 3.00, 10.00, 0.00, -2, 1, -1 ] + 4: [ 1, 1, 1, 2.50, 7.00, 3.00, 10.00, 0.00, -2, 1, -1 ] + 5: [ 1, 1, 1, 0.55, 3.50, 1.00, 10.00, 0.00, -2, 1, -1 ] + 6: [ 1, 1, 1, 0.30, 3.00, 1.00, 10.00, 0.00, -2, 1, -1 ] + 7: [ 1, 1, 1, 0.30, 0.80, 1.00, 10.00, 0.00, -2, 1, -1 ] + 8: [ 1, 1, 1, 0.40, 0.40, 1.00, 10.00, 0.00, -2, 1, -1 ] + 9: [ 1, 1, 1, 0.40, 0.40, 1.00, 10.00, 0.00, -2, 1, -1 ] + 10: [ 1, 1, 1, 0.40, 0.40, 1.00, 10.00, 0.00, -2, 1, -1 ] + 11: [ 1, 1, 1, 0.45, 0.45, 1.00, 10.00, 0.00, -2, 1, -1 ] + 12: [ 1, 1, 1, 0.45, 0.45, 1.00, 10.00, 0.00, -2, 1, -1 ] + 13: [ 1, 1, 1, 0.55, 0.55, 1.00, 10.00, 0.00, -2, 1, -1 ] + 14: [ 1, 1, 1, 0.80, 0.80, 2.00, 10.00, 0.00, -2, 1, -1 ] + 15: [ 1, 1, 0, 4.00, 4.00, 4.50, 10.00, 0.00, -2, 1, -1 ] + 16: [ 1, 1, 1, 4.00, 19.00, 4.50, 10.00, 0.00, -2, 1, -1 ] + 17: [ 1, 1, 1, 4.00, 30.00, 2.00, 10.00, 0.00, -2, 1, -1 ] + 18: [ 1, 1, 1, 3.50, 25.00, 2.00, 10.00, 0.00, -2, 1, -1 ] + 19: [ 1, 1, 1, 3.00, 16.50, 2.00, 10.00, 0.00, -2, 1, -1 ] + 20: [ 1, 1, 1, 3.00, 12.00, 2.00, 10.00, 0.00, -2, 1, -1 ] + 21: [ 1, 1, 1, 3.00, 9.00, 2.00, 10.00, 0.00, -2, 1, -1 ] + 22: [ 1, 1, 1, 3.00, 6.50, 2.00, 10.00, 0.00, -2, 1, -1 ] diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/cris-fsr_n20.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/cris-fsr_n20.yaml new file mode 100644 index 000000000..a5c4ab759 --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/atmosphere/cris-fsr_n20.yaml @@ -0,0 +1,452 @@ +# Instrument metadata +# ------------------- +commissioned: 2018-02-17T00:00:00 + +observer_type: satellite # Type of chronicle to use + +# Instrument initial configuration +# -------------------------------- +channel_variables: + simulated: min + active: min + biascorrtd: min + error: max + error_cld: max + ermax: max + var_b: min + var_pg: min + icld_det: min + icloud: min + iaerosol: min +channel_values: + 19: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 24: [ 1, 1, 1, 0.700, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 26: [ 1, 1, 1, 0.700, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 27: [ 1, -1, 1, 0.700, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 28: [ 1, 1, 1, 0.700, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 31: [ 1, -1, 1, 1.359, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 32: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 33: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 37: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 39: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 42: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 44: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 47: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 49: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 50: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 51: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 52: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 53: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 54: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 55: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 56: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 57: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 58: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 59: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 60: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 61: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 62: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 63: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 64: [ 1, -1, 1, 0.756, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 65: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 66: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 67: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 68: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 69: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 70: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 71: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 72: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 73: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 74: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 75: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 76: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 77: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 78: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 79: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 80: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 81: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 82: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 83: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 84: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 85: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 86: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 87: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 88: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 89: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 90: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 91: [ 1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 92: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 93: [ 1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 94: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 95: [ 1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 96: [ 1, -1, 1, 0.635, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 97: [ 1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 98: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 99: [ 1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 100: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 101: [ 1, -1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 102: [ 1, -1, 1, 0.735, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 103: [ 1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 104: [ 1, -1, 1, 0.878, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 105: [ 1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 106: [ 1, -1, 1, 0.696, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 107: [ 1, 1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 108: [ 1, -1, 1, 2.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 109: [ 1, 1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 110: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 111: [ 1, 1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 112: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 113: [ 1, 1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 114: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 115: [ 1, 1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 116: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 117: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 118: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 119: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 120: [ 1, -1, 1, 0.701, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 121: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 122: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 123: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 124: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 125: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 126: [ 1, -1, 1, 0.663, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 127: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 128: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 129: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 130: [ 1, -1, 1, 1.083, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 131: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 132: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 133: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 134: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 135: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 136: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 137: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 138: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 139: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 140: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 141: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 142: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 143: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 144: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 145: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 146: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 147: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 148: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 149: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 150: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 151: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 152: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 153: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 154: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 155: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 156: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 157: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 158: [ 1, -1, 1, 0.773, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 159: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 160: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 161: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 162: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 163: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 164: [ 1, -1, 1, 0.813, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 165: [ 1, -1, 1, 0.907, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 166: [ 1, -1, 1, 0.802, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 167: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 168: [ 1, -1, 1, 1.493, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 169: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 170: [ 1, -1, 1, 0.856, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 171: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 172: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 173: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 174: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 175: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 176: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 177: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 178: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 179: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 180: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 181: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 182: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 183: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 184: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 185: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 186: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 187: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 188: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 189: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 190: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 191: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 192: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 193: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 194: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 195: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 196: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 197: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 198: [ 1, -1, 1, 2.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 199: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 200: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 208: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 211: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 216: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 224: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 234: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 236: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 238: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 239: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 242: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 246: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 248: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 255: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 264: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 266: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 268: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 275: [ 1, 1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 279: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 283: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 285: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 291: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 295: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 301: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 305: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 311: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 332: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 342: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 389: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 400: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 402: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 404: [ 1, -1, 1, 2.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 406: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 410: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 427: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 439: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 440: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 441: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 445: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 449: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 455: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 458: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 461: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 464: [ 1, 1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 467: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 470: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 473: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 475: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 482: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 486: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 487: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 490: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 493: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 496: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 499: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 501: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 503: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 505: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 511: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 513: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 514: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 518: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 519: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 520: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 522: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 529: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 534: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 563: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 568: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 575: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 592: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 594: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 596: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 598: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 600: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 602: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 604: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 611: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 614: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 616: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 618: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 620: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 622: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 626: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 631: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 638: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 646: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 648: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 652: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 659: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 673: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 675: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 678: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 684: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 688: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 694: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 700: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 707: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 710: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 713: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 714: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 718: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 720: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 722: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 725: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 728: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 735: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 742: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 748: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 753: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 762: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 780: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 784: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 798: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 849: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 860: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 862: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 866: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 874: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 882: [ 1, 1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 890: [ 1, 1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 898: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 906: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 907: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 908: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 914: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 937: [ 1, 1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 972: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 973: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 978: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 980: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 981: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 988: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 995: [ 1, 1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 998: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1000: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1003: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1008: [ 1, 1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 1009: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1010: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1014: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1017: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1018: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1020: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1022: [ 1, 1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 1024: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1026: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1029: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1030: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1032: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1034: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1037: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1038: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1041: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1042: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1044: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1046: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1049: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1050: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1053: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1054: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1058: [ 1, 1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 1060: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1062: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1064: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1066: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1069: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1076: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1077: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1080: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1086: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1091: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1095: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1101: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1109: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1112: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1121: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1128: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1133: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1163: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1172: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1187: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1189: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1205: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1211: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1219: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1231: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1245: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1271: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1289: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1300: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1313: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1316: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1325: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1329: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1346: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1347: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1473: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1474: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1491: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1499: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1553: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1570: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1596: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1602: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1619: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1624: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1635: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1939: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1940: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1941: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1942: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1943: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1944: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1945: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1946: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1947: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1948: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1949: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1950: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1951: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1952: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1953: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1954: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1955: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1956: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1957: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1958: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1959: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1960: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1961: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1962: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1963: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1964: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1965: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1966: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1967: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1968: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1969: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1970: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1971: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1972: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1973: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1974: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1975: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1976: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1977: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1978: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1979: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1980: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1981: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1982: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1983: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1984: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1985: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1986: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1987: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2119: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2140: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2143: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2147: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2153: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2158: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2161: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2168: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2171: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2175: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2182: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/cris-fsr_npp.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/cris-fsr_npp.yaml new file mode 100644 index 000000000..638bcef33 --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/atmosphere/cris-fsr_npp.yaml @@ -0,0 +1,452 @@ +# Instrument metadata +# ------------------- +commissioned: 2017-10-20T00:00:00 + +observer_type: satellite # Type of chronicle to use + +# Instrument initial configuration +# -------------------------------- +channel_variables: + simulated: min + active: min + biascorrtd: min + error: max + error_cld: max + ermax: max + var_b: min + var_pg: min + icld_det: min + icloud: min + iaerosol: min +channel_values: + 19: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 24: [ 1, -1, 1, 0.700, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 26: [ 1, -1, 1, 0.700, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 27: [ 1, -1, 1, 0.700, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 28: [ 1, -1, 1, 0.700, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 31: [ 1, -1, 1, 1.359, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 32: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 33: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 37: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 39: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 42: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 44: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 47: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 49: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 50: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 51: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 52: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 53: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 54: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 55: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 56: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 57: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 58: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 59: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 60: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 61: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 62: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 63: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 64: [ 1, -1, 1, 0.756, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 65: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 66: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 67: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 68: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 69: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 70: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 71: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 72: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 73: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 74: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 75: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 76: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 77: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 78: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 79: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 80: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 81: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 82: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 83: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 84: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 85: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 86: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 87: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 88: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 89: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 90: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 91: [ 1, -1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 92: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 93: [ 1, -1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 94: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 95: [ 1, -1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 96: [ 1, -1, 1, 0.635, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 97: [ 1, -1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 98: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 99: [ 1, -1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 100: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 101: [ 1, -1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 102: [ 1, -1, 1, 0.735, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 103: [ 1, -1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 104: [ 1, -1, 1, 0.878, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 105: [ 1, -1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 106: [ 1, -1, 1, 0.696, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 107: [ 1, -1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 108: [ 1, -1, 1, 2.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 109: [ 1, -1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 110: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 111: [ 1, -1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 112: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 113: [ 1, -1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 114: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 115: [ 1, -1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 116: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 117: [ 1, -1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 118: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 119: [ 1, -1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 120: [ 1, -1, 1, 0.701, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 121: [ 1, -1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 122: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 123: [ 1, -1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 124: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 125: [ 1, -1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 126: [ 1, -1, 1, 0.663, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 127: [ 1, -1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 128: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 129: [ 1, -1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 130: [ 1, -1, 1, 1.083, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 131: [ 1, -1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 132: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 133: [ 1, -1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 134: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 135: [ 1, -1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 136: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 137: [ 1, -1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 138: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 139: [ 1, -1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 140: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 141: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 142: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 143: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 144: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 145: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 146: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 147: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 148: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 149: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 150: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 151: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 152: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 153: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 154: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 155: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 156: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 157: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 158: [ 1, -1, 1, 0.773, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 159: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 160: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 161: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 162: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 163: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 164: [ 1, -1, 1, 0.813, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 165: [ 1, -1, 1, 0.907, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 166: [ 1, -1, 1, 0.802, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 167: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 168: [ 1, -1, 1, 1.493, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 169: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 170: [ 1, -1, 1, 0.856, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 171: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 172: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 173: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 174: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 175: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 176: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 177: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 178: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 179: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 180: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 181: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 182: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 183: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 184: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 185: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 186: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 187: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 188: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 189: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 190: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 191: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 192: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 193: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 194: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 195: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 196: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 197: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 198: [ 1, -1, 1, 2.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 199: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 200: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 208: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 211: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 216: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 224: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 234: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 236: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 238: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 239: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 242: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 246: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 248: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 255: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 264: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 266: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 268: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 275: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 279: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 283: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 285: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 291: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 295: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 301: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 305: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 311: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 332: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 342: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 389: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 400: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 402: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 404: [ 1, -1, 1, 2.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 406: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 410: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 427: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 439: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 440: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 441: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 445: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 449: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 455: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 458: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 461: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 464: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 467: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 470: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 473: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 475: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 482: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 486: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 487: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 490: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 493: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 496: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 499: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 501: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 503: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 505: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 511: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 513: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 514: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 518: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 519: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 520: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 522: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 529: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 534: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 563: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 568: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 575: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 592: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 594: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 596: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 598: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 600: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 602: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 604: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 611: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 614: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 616: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 618: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 620: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 622: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 626: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 631: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 638: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 646: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 648: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 652: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 659: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 673: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 675: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 678: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 684: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 688: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 694: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 700: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 707: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 710: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 713: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 714: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 718: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 720: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 722: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 725: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 728: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 735: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 742: [ 0, -1, 0, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 748: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 753: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 762: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 780: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 784: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 798: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 849: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 860: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 862: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 866: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 874: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 882: [ 0, -1, 0, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 890: [ 0, -1, 0, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 898: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 906: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 907: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 908: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 914: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 937: [ 0, -1, 0, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 972: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 973: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 978: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 980: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 981: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 988: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 995: [ 0, -1, 0, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 998: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1000: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1003: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1008: [ 0, -1, 0, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 1009: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1010: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1014: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1017: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1018: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1020: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1022: [ 0, -1, 0, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 1024: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1026: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1029: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1030: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1032: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1034: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1037: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1038: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1041: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1042: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1044: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1046: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1049: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1050: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1053: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1054: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1058: [ 0, -1, 0, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 1060: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1062: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1064: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1066: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1069: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1076: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1077: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1080: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1086: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1091: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1095: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1101: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1109: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1112: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1121: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1128: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1133: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1163: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1172: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1187: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1189: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1205: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1211: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1219: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1231: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1245: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1271: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1289: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1300: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1313: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1316: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1325: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1329: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1346: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1347: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1473: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1474: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1491: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1499: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1553: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1570: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1596: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1602: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1619: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1624: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1635: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1939: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1940: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1941: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1942: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1943: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1944: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1945: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1946: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1947: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1948: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1949: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1950: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1951: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1952: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1953: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1954: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1955: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1956: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1957: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1958: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1959: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1960: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1961: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1962: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1963: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1964: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1965: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1966: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1967: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1968: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1969: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1970: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1971: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1972: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1973: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1974: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1975: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1976: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1977: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1978: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1979: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1980: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1981: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1982: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1983: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1984: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1985: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1986: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1987: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2119: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2140: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2143: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2147: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2153: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2158: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2161: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2168: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2171: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2175: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2182: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/iasi_metop-a.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/iasi_metop-a.yaml new file mode 100644 index 000000000..90dc0d74b --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/atmosphere/iasi_metop-a.yaml @@ -0,0 +1,638 @@ +# Instrument metadata +# ------------------- +commissioned: 2008-08-01T00:00:00 +decommissioned: 2021-09-08T06:00:00 + +observer_type: satellite # Type of chronicle to use + +# Instrument initial configuration +# -------------------------------- +channel_variables: + simulated: min + active: min + biascorrtd: min + error: max + error_cld: max + ermax: max + var_b: min + var_pg: min + icld_det: min + icloud: min + iaerosol: min +channel_values: + 16: [ 1, -1, 1, 1.38, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 29: [ 1, -1, 1, 0.81, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 32: [ 1, -1, 1, 0.75, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 35: [ 1, -1, 1, 0.79, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 38: [ 1, -1, 1, 0.72, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 41: [ 1, -1, 1, 0.74, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 44: [ 1, -1, 1, 0.68, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 47: [ 1, -1, 1, 0.72, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 49: [ 1, -1, 1, 0.65, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 50: [ 1, -1, 1, 0.65, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 51: [ 1, -1, 1, 0.65, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 53: [ 1, -1, 1, 0.69, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 55: [ 1, -1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 56: [ 1, -1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 57: [ 1, -1, 1, 0.65, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 59: [ 1, -1, 1, 0.67, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 61: [ 1, -1, 1, 0.62, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 62: [ 1, -1, 1, 0.61, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 63: [ 1, -1, 1, 0.62, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 66: [ 1, -1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 68: [ 1, -1, 1, 0.59, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 70: [ 1, -1, 1, 0.76, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 72: [ 1, -1, 1, 1.22, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 74: [ 1, -1, 1, 0.78, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 76: [ 1, -1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 78: [ 1, -1, 1, 0.62, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 79: [ 1, -1, 1, 0.61, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 81: [ 1, -1, 1, 0.69, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 82: [ 1, -1, 1, 0.65, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 83: [ 1, -1, 1, 0.59, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 84: [ 1, -1, 1, 0.61, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 85: [ 1, -1, 1, 0.59, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 86: [ 1, -1, 1, 0.68, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 87: [ 1, -1, 1, 0.62, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 89: [ 1, -1, 1, 0.68, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 92: [ 1, -1, 1, 4.38, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 93: [ 1, -1, 1, 3.05, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 95: [ 1, -1, 1, 2.31, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 97: [ 1, -1, 1, 1.56, 0.00, 4.00, 10.00, 0.00, -1, -1, -1 ] + 99: [ 1, -1, 1, 1.33, 0.00, 4.00, 10.00, 0.00, -1, -1, -1 ] + 101: [ 1, -1, 1, 1.58, 0.00, 3.50, 10.00, 0.00, -1, -1, -1 ] + 103: [ 1, -1, 1, 0.93, 0.00, 2.50, 10.00, 0.00, -1, -1, -1 ] + 104: [ 1, -1, 1, 1.67, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 106: [ 1, -1, 1, 0.72, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 109: [ 1, -1, 1, 0.57, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 110: [ 1, -1, 1, 0.58, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 111: [ 1, -1, 1, 0.55, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 113: [ 1, -1, 1, 0.68, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 116: [ 1, -1, 1, 0.59, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 119: [ 1, -1, 1, 0.68, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 122: [ 1, -1, 1, 0.59, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 125: [ 1, -1, 1, 0.65, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 128: [ 1, -1, 1, 0.58, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 131: [ 1, -1, 1, 0.62, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 133: [ 1, -1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 135: [ 1, -1, 1, 0.58, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 138: [ 1, -1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 141: [ 1, -1, 1, 0.55, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 144: [ 1, -1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 146: [ 1, -1, 1, 0.50, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 148: [ 1, -1, 1, 0.82, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 150: [ 1, -1, 1, 0.59, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 151: [ 1, -1, 1, 0.62, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 154: [ 1, -1, 1, 0.51, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 157: [ 1, -1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 159: [ 1, -1, 1, 0.52, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 160: [ 1, -1, 1, 0.51, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 161: [ 1, -1, 1, 0.51, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 163: [ 1, -1, 1, 0.76, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 167: [ 1, -1, 1, 0.52, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 170: [ 1, -1, 1, 0.57, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 173: [ 1, -1, 1, 0.55, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 176: [ 1, -1, 1, 0.69, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 179: [ 1, -1, 1, 0.58, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 180: [ 1, -1, 1, 0.65, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 185: [ 1, -1, 1, 0.61, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 187: [ 1, -1, 1, 0.59, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 191: [ 1, -1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 193: [ 1, -1, 1, 0.76, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 197: [ 1, -1, 1, 0.72, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 199: [ 1, -1, 1, 1.05, 0.00, 3.50, 10.00, 0.00, 1, -1, -1 ] + 200: [ 1, -1, 1, 0.75, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 202: [ 1, -1, 1, 0.51, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 203: [ 1, -1, 1, 0.65, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 205: [ 1, -1, 1, 1.30, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 207: [ 1, -1, 1, 0.69, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 210: [ 1, -1, 1, 0.93, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 212: [ 1, -1, 1, 1.49, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 213: [ 1, -1, 1, 1.12, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 214: [ 1, -1, 1, 0.68, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 217: [ 1, -1, 1, 0.66, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 218: [ 1, -1, 1, 0.67, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 219: [ 1, -1, 1, 0.59, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 222: [ 1, -1, 1, 0.59, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 224: [ 1, -1, 1, 0.69, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 225: [ 1, -1, 1, 0.67, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 226: [ 1, -1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 228: [ 1, -1, 1, 0.62, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 230: [ 1, -1, 1, 0.72, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 231: [ 1, -1, 1, 0.69, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 232: [ 1, -1, 1, 0.66, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 236: [ 1, -1, 1, 0.79, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 237: [ 1, -1, 1, 0.78, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 239: [ 1, -1, 1, 0.74, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 243: [ 1, -1, 1, 0.88, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 246: [ 1, -1, 1, 0.77, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 249: [ 1, -1, 1, 0.88, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 252: [ 1, -1, 1, 0.86, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 254: [ 1, -1, 1, 1.00, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 259: [ 1, -1, 1, 0.87, 0.00, 2.50, 10.00, 0.00, -1, -1, -1 ] + 260: [ 1, -1, 1, 0.85, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 262: [ 1, -1, 1, 0.88, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 265: [ 1, -1, 1, 0.84, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 267: [ 1, -1, 1, 0.84, 0.00, 3.50, 10.00, 0.00, 1, -1, -1 ] + 269: [ 1, -1, 1, 0.84, 0.00, 2.50, 10.00, 0.00, -1, -1, -1 ] + 275: [ 1, -1, 1, 0.80, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 279: [ 1, -1, 1, 0.80, 0.00, 3.00, 10.00, 0.00, -1, -1, -1 ] + 282: [ 1, -1, 1, 0.87, 0.00, 3.50, 10.00, 0.00, 1, -1, -1 ] + 285: [ 1, -1, 1, 0.98, 0.00, 3.00, 10.00, 0.00, -1, -1, -1 ] + 294: [ 1, -1, 1, 0.52, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 296: [ 1, -1, 1, 0.65, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 299: [ 1, -1, 1, 0.69, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 300: [ 1, -1, 1, 0.61, 0.00, 4.00, 10.00, 0.00, -1, -1, -1 ] + 303: [ 1, -1, 1, 0.60, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 306: [ 1, -1, 1, 0.67, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 309: [ 1, -1, 1, 0.79, 0.00, 4.50, 10.00, 0.00, -1, -1, -1 ] + 313: [ 1, -1, 1, 0.62, 0.00, 4.50, 10.00, 0.00, -1, -1, -1 ] + 320: [ 1, -1, 1, 0.66, 0.00, 4.50, 10.00, 0.00, -1, -1, -1 ] + 323: [ 1, -1, 1, 0.70, 0.00, 4.50, 10.00, 0.00, 1, -1, -1 ] + 326: [ 1, -1, 1, 0.65, 0.00, 4.50, 10.00, 0.00, -1, -1, -1 ] + 327: [ 1, -1, 1, 0.62, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 329: [ 1, -1, 1, 0.61, 0.00, 4.50, 10.00, 0.00, 1, -1, -1 ] + 332: [ 1, -1, 1, 0.62, 0.00, 4.00, 10.00, 0.00, -1, -1, -1 ] + 335: [ 1, -1, 1, 0.53, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 345: [ 1, -1, 1, 0.60, 0.00, 4.50, 10.00, 0.00, 1, -1, -1 ] + 347: [ 1, -1, 1, 0.68, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 350: [ 1, -1, 1, 0.95, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 354: [ 1, -1, 1, 0.63, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 356: [ 1, -1, 1, 0.97, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 360: [ 1, -1, 1, 0.65, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 363: [ 1, -1, 1, 0.98, 0.00, 3.00, 10.00, 0.00, -1, -1, -1 ] + 366: [ 1, -1, 1, 0.58, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 371: [ 1, -1, 1, 0.73, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 372: [ 1, -1, 1, 0.65, 0.00, 2.50, 10.00, 0.00, -1, -1, -1 ] + 373: [ 1, -1, 1, 0.85, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 375: [ 1, -1, 1, 0.99, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 377: [ 1, -1, 1, 0.76, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 379: [ 1, -1, 1, 0.85, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 381: [ 1, -1, 1, 0.97, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 383: [ 1, -1, 1, 0.77, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 386: [ 1, -1, 1, 0.62, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 389: [ 1, -1, 1, 0.63, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 398: [ 1, -1, 1, 1.21, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 401: [ 1, -1, 1, 1.41, 0.00, 4.50, 10.00, 0.00, 1, -1, -1 ] + 404: [ 1, -1, 1, 1.55, 0.00, 4.50, 10.00, 0.00, 1, -1, -1 ] + 405: [ 1, -1, 1, 1.78, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 407: [ 1, -1, 1, 1.35, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 408: [ 1, -1, 1, 1.14, 0.00, 4.00, 10.00, 0.00, -1, -1, -1 ] + 410: [ 1, -1, 1, 1.69, 0.00, 5.00, 10.00, 0.00, 1, -1, -1 ] + 411: [ 1, -1, 1, 1.79, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 414: [ 1, -1, 1, 1.46, 0.00, 5.00, 10.00, 0.00, 1, -1, -1 ] + 416: [ 1, -1, 1, 1.63, 0.00, 5.00, 10.00, 0.00, 1, -1, -1 ] + 418: [ 1, -1, 1, 1.94, 0.00, 5.50, 10.00, 0.00, -1, -1, -1 ] + 423: [ 1, -1, 1, 2.01, 0.00, 5.50, 10.00, 0.00, -1, -1, -1 ] + 426: [ 1, -1, 1, 1.24, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 428: [ 1, -1, 1, 1.76, 0.00, 5.00, 10.00, 0.00, 1, -1, -1 ] + 432: [ 1, -1, 1, 1.26, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 433: [ 1, -1, 1, 1.47, 0.00, 4.50, 10.00, 0.00, -1, -1, -1 ] + 434: [ 1, -1, 1, 1.90, 0.00, 5.50, 10.00, 0.00, 1, -1, -1 ] + 439: [ 1, -1, 1, 1.66, 0.00, 5.50, 10.00, 0.00, 1, -1, -1 ] + 442: [ 1, -1, 1, 2.13, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 445: [ 1, -1, 1, 1.49, 0.00, 4.50, 10.00, 0.00, 1, -1, -1 ] + 450: [ 1, -1, 1, 1.52, 0.00, 4.50, 10.00, 0.00, -1, -1, -1 ] + 457: [ 1, -1, 1, 1.55, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 459: [ 1, -1, 1, 1.96, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 472: [ 1, -1, 1, 2.31, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 477: [ 1, -1, 1, 2.33, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 483: [ 1, -1, 1, 2.32, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 509: [ 1, -1, 1, 2.31, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 515: [ 1, -1, 1, 2.33, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 546: [ 1, -1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 552: [ 1, -1, 1, 2.33, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 559: [ 1, -1, 1, 1.84, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 566: [ 1, -1, 1, 2.29, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 571: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 573: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 578: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 584: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 594: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 625: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 646: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 662: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 668: [ 1, -1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 705: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 739: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 756: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 797: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 867: [ 1, -1, 1, 2.30, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 906: [ 1, -1, 1, 2.15, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 921: [ 1, -1, 1, 2.31, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1027: [ 1, -1, 1, 2.37, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1046: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1090: [ 1, -1, 1, 2.29, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1098: [ 1, -1, 1, 2.29, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1121: [ 1, -1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1133: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1173: [ 1, -1, 1, 2.32, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1191: [ 1, -1, 1, 2.32, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1194: [ 1, -1, 1, 2.31, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1222: [ 1, -1, 1, 2.32, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1271: [ 1, -1, 1, 2.32, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1283: [ 1, -1, 1, 2.31, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1338: [ 1, -1, 1, 2.31, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1409: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1414: [ 1, -1, 1, 2.29, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1420: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1424: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1427: [ 1, -1, 1, 2.29, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1430: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1434: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1440: [ 1, -1, 1, 2.25, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1442: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1445: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1450: [ 1, -1, 1, 2.21, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1454: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1460: [ 1, -1, 1, 2.17, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1463: [ 1, -1, 1, 2.18, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1469: [ 1, -1, 1, 2.17, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1474: [ 1, -1, 1, 2.21, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1479: [ 1, -1, 1, 1.99, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1483: [ 1, -1, 1, 2.16, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1487: [ 1, -1, 1, 2.20, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1494: [ 1, -1, 1, 2.13, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1496: [ 1, -1, 1, 2.12, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1502: [ 1, -1, 1, 2.13, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1505: [ 1, -1, 1, 2.10, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1509: [ 1, -1, 1, 2.12, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1510: [ 1, -1, 1, 2.11, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1513: [ 1, -1, 1, 2.09, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1518: [ 1, -1, 1, 2.09, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1521: [ 1, -1, 1, 2.08, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1526: [ 1, -1, 1, 2.09, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1529: [ 1, -1, 1, 2.04, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1532: [ 1, -1, 1, 2.04, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1536: [ 1, -1, 1, 2.10, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1537: [ 1, -1, 1, 2.01, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1541: [ 1, -1, 1, 2.05, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1545: [ 1, -1, 1, 2.03, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1548: [ 1, -1, 1, 2.06, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1553: [ 1, -1, 1, 1.98, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1560: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1568: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1574: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1579: [ 1, -1, 1, 1.70, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1583: [ 1, -1, 1, 1.76, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1585: [ 1, -1, 1, 1.77, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1587: [ 1, -1, 1, 1.83, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1606: [ 1, -1, 1, 2.04, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1626: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1639: [ 1, -1, 1, 1.99, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1643: [ 1, -1, 1, 1.99, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1652: [ 1, -1, 1, 2.07, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1658: [ 1, -1, 1, 2.02, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1659: [ 1, -1, 1, 2.04, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1666: [ 1, -1, 1, 2.10, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1671: [ 1, -1, 1, 2.06, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1675: [ 1, -1, 1, 2.18, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1681: [ 1, -1, 1, 2.21, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1694: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1697: [ 1, -1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1710: [ 1, -1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1786: [ 1, -1, 1, 1.98, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1791: [ 1, -1, 1, 2.20, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1805: [ 1, -1, 1, 2.18, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1839: [ 1, -1, 1, 2.18, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1884: [ 1, -1, 1, 2.21, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1913: [ 1, -1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1946: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1947: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1991: [ 1, -1, 1, 2.25, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2019: [ 1, -1, 1, 1.80, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2094: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2119: [ 1, -1, 1, 1.73, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2213: [ 1, -1, 1, 1.73, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2239: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2271: [ 1, -1, 1, 1.67, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2289: [ 1, -1, 1, 2.21, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2321: [ 1, -1, 1, 1.72, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2333: [ 1, -1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2346: [ 1, -1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2349: [ 1, -1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2352: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2359: [ 1, -1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2367: [ 1, -1, 1, 2.12, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2374: [ 1, -1, 1, 2.17, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2398: [ 1, -1, 1, 1.74, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2426: [ 1, -1, 1, 2.02, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2562: [ 1, -1, 1, 1.88, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2701: [ 1, -1, 1, 1.67, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2741: [ 1, -1, 1, 1.73, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2745: [ 1, -1, 1, 1.83, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2760: [ 1, -1, 1, 1.82, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2819: [ 1, -1, 1, 1.73, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2889: [ 1, -1, 1, 1.83, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2907: [ 1, -1, 1, 2.19, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2910: [ 1, -1, 1, 1.84, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2919: [ 1, -1, 1, 1.89, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2921: [ 1, -1, 1, 1.60, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2939: [ 1, -1, 1, 1.71, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2944: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2945: [ 1, -1, 1, 1.85, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2948: [ 1, -1, 1, 1.84, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2951: [ 1, -1, 1, 1.87, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2958: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2971: [ 1, -1, 1, 1.52, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2977: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2985: [ 1, -1, 1, 1.87, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2988: [ 1, -1, 1, 1.89, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2990: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2991: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2993: [ 1, -1, 1, 1.93, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3002: [ 1, -1, 1, 1.90, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3008: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3014: [ 1, -1, 1, 1.90, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3027: [ 1, -1, 1, 1.89, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3029: [ 1, -1, 1, 1.89, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3030: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3036: [ 1, -1, 1, 1.90, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3047: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3049: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3052: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3053: [ 1, -1, 1, 1.93, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3055: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3058: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3064: [ 1, -1, 1, 1.92, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3069: [ 1, -1, 1, 1.77, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3087: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3093: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3098: [ 1, -1, 1, 1.19, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3105: [ 1, -1, 1, 1.96, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3107: [ 1, -1, 1, 1.98, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3110: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3116: [ 1, -1, 1, 1.55, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3127: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3129: [ 1, -1, 1, 1.92, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3136: [ 1, -1, 1, 1.92, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3146: [ 1, -1, 1, 1.97, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3151: [ 1, -1, 1, 1.93, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3160: [ 1, -1, 1, 1.99, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3165: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3168: [ 1, -1, 1, 1.12, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3175: [ 1, -1, 1, 1.93, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3178: [ 1, -1, 1, 1.92, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3189: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3207: [ 1, -1, 1, 1.85, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3228: [ 1, -1, 1, 1.84, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3244: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3248: [ 1, -1, 1, 1.12, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3252: [ 1, -1, 1, 1.82, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3256: [ 1, -1, 1, 1.82, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3263: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3281: [ 1, -1, 1, 1.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3295: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3303: [ 1, -1, 1, 1.96, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3309: [ 1, -1, 1, 1.21, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3312: [ 1, -1, 1, 1.83, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3322: [ 1, -1, 1, 1.96, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3326: [ 1, -1, 1, 1.36, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3354: [ 1, -1, 1, 1.96, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3366: [ 1, -1, 1, 1.82, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3375: [ 1, -1, 1, 1.92, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3378: [ 1, -1, 1, 1.68, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3411: [ 1, -1, 1, 1.93, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3416: [ 1, -1, 1, 1.23, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3432: [ 1, -1, 1, 1.96, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3438: [ 1, -1, 1, 1.93, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3440: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3442: [ 1, -1, 1, 1.41, 0.00, 5.50, 10.00, 0.00, -1, -1, -1 ] + 3444: [ 1, -1, 1, 1.16, 0.00, 4.50, 10.00, 0.00, -1, -1, -1 ] + 3446: [ 1, -1, 1, 1.60, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3448: [ 1, -1, 1, 1.25, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 3450: [ 1, -1, 1, 1.20, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 3452: [ 1, -1, 1, 1.65, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3454: [ 1, -1, 1, 1.66, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3458: [ 1, -1, 1, 1.87, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3467: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3476: [ 1, -1, 1, 1.96, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3484: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3491: [ 1, -1, 1, 1.25, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 3497: [ 1, -1, 1, 1.93, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3499: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3504: [ 1, -1, 1, 1.70, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3506: [ 1, -1, 1, 0.99, 0.00, 4.00, 10.00, 0.00, -1, -1, -1 ] + 3509: [ 1, -1, 1, 1.81, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3518: [ 1, -1, 1, 1.92, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3527: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3555: [ 1, -1, 1, 1.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3575: [ 1, -1, 1, 1.47, 0.00, 4.50, 10.00, 0.00, -1, -1, -1 ] + 3577: [ 1, -1, 1, 1.15, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3580: [ 1, -1, 1, 1.58, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3582: [ 1, -1, 1, 1.18, 0.00, 4.50, 10.00, 0.00, -1, -1, -1 ] + 3586: [ 1, -1, 1, 1.82, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3589: [ 1, -1, 1, 1.13, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3599: [ 1, -1, 1, 1.83, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3610: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3626: [ 1, -1, 1, 1.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3638: [ 1, -1, 1, 1.27, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 3646: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3653: [ 1, -1, 1, 1.45, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3658: [ 1, -1, 1, 1.60, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3661: [ 1, -1, 1, 1.29, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 3673: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3689: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3700: [ 1, -1, 1, 1.23, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 3710: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3726: [ 1, -1, 1, 1.21, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 3763: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3814: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3841: [ 1, -1, 1, 1.90, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3888: [ 1, -1, 1, 1.33, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 4032: [ 1, -1, 1, 1.75, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4059: [ 1, -1, 1, 2.02, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4068: [ 1, -1, 1, 1.98, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4082: [ 1, -1, 1, 2.03, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4095: [ 1, -1, 1, 1.83, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4160: [ 1, -1, 1, 1.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4234: [ 1, -1, 1, 2.04, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4257: [ 1, -1, 1, 2.02, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4411: [ 1, -1, 1, 1.90, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4498: [ 1, -1, 1, 2.00, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4520: [ 1, -1, 1, 2.02, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4552: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4567: [ 1, -1, 1, 1.93, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4608: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4646: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4698: [ 1, -1, 1, 1.99, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4808: [ 1, -1, 1, 2.00, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4849: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4920: [ 1, -1, 1, 1.96, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4939: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4947: [ 1, -1, 1, 1.92, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4967: [ 1, -1, 1, 1.88, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4991: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4996: [ 1, -1, 1, 1.84, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5015: [ 1, -1, 1, 1.87, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5028: [ 1, -1, 1, 1.77, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5056: [ 1, -1, 1, 1.89, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5128: [ 1, -1, 1, 1.89, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5130: [ 1, -1, 1, 1.88, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5144: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5170: [ 1, -1, 1, 1.82, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5178: [ 1, -1, 1, 1.79, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5183: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5188: [ 1, -1, 1, 2.06, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5191: [ 1, -1, 1, 2.33, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5368: [ 1, -1, 1, 1.88, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5371: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5379: [ 1, -1, 1, 1.81, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5381: [ 1, -1, 1, 1.80, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5383: [ 1, -1, 1, 1.80, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5397: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5399: [ 1, -1, 1, 1.90, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5401: [ 1, -1, 1, 2.00, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5403: [ 1, -1, 1, 2.06, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5405: [ 1, -1, 1, 2.10, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5446: [ 1, -1, 1, 2.20, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5455: [ 1, -1, 1, 2.00, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5472: [ 1, -1, 1, 2.16, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5480: [ 1, -1, 1, 1.98, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5483: [ 1, -1, 1, 1.80, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5485: [ 1, -1, 1, 1.80, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5492: [ 1, -1, 1, 1.85, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5497: [ 1, -1, 1, 1.75, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5502: [ 1, -1, 1, 2.04, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5507: [ 1, -1, 1, 2.19, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5509: [ 1, -1, 1, 2.14, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5517: [ 1, -1, 1, 2.19, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5528: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5558: [ 1, -1, 1, 2.10, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5697: [ 1, -1, 1, 2.11, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5714: [ 1, -1, 1, 2.18, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5749: [ 1, -1, 1, 2.03, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5766: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5785: [ 1, -1, 1, 2.19, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5798: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5799: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5801: [ 1, -1, 1, 2.21, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5817: [ 1, -1, 1, 2.21, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5833: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5834: [ 1, -1, 1, 2.33, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5836: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5849: [ 1, -1, 1, 2.21, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5851: [ 1, -1, 1, 2.12, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5852: [ 1, -1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5865: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5869: [ 1, -1, 1, 2.25, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5881: [ 1, -1, 1, 1.88, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5884: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5897: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5900: [ 1, -1, 1, 2.36, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5916: [ 1, -1, 1, 2.29, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5932: [ 1, -1, 1, 2.35, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5948: [ 1, -1, 1, 2.30, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5963: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5968: [ 1, -1, 1, 2.08, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5978: [ 1, -1, 1, 2.05, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5988: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5992: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5994: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5997: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6003: [ 1, -1, 1, 1.97, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6008: [ 1, -1, 1, 2.25, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6023: [ 1, -1, 1, 2.25, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6026: [ 1, -1, 1, 2.25, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6039: [ 1, -1, 1, 2.31, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6053: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6056: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6067: [ 1, -1, 1, 2.13, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6071: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6082: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6085: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6098: [ 1, -1, 1, 2.41, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6112: [ 1, -1, 1, 2.34, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6126: [ 1, -1, 1, 9.32, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6135: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6140: [ 1, -1, 1, 2.38, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6149: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6154: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6158: [ 1, -1, 1, 2.39, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6161: [ 1, -1, 1, 2.11, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6168: [ 1, -1, 1, 2.09, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6174: [ 1, -1, 1, 2.10, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6182: [ 1, -1, 1, 2.06, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6187: [ 1, -1, 1, 2.12, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6205: [ 1, -1, 1, 2.08, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6209: [ 1, -1, 1, 2.00, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6213: [ 1, -1, 1, 1.93, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6317: [ 1, -1, 1, 2.02, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6339: [ 1, -1, 1, 2.55, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6342: [ 1, -1, 1, 1.54, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6366: [ 1, -1, 1, 1.64, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6381: [ 1, -1, 1, 1.51, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6391: [ 1, -1, 1, 1.55, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6489: [ 1, -1, 1, 2.82, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6962: [ 1, -1, 1, 2.92, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6966: [ 1, -1, 1, 2.55, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6970: [ 1, -1, 1, 2.37, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6975: [ 1, -1, 1, 1.85, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6977: [ 1, -1, 1, 1.60, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6982: [ 1, -1, 1, 1.72, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6985: [ 1, -1, 1, 1.74, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6987: [ 1, -1, 1, 1.79, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6989: [ 1, -1, 1, 1.90, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6991: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6993: [ 1, -1, 1, 2.00, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6995: [ 1, -1, 1, 2.04, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6997: [ 1, -1, 1, 2.08, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6999: [ 1, -1, 1, 2.12, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7000: [ 1, -1, 1, 2.13, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7004: [ 1, -1, 1, 2.16, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7008: [ 1, -1, 1, 2.18, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7013: [ 1, -1, 1, 2.18, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7016: [ 1, -1, 1, 2.20, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7021: [ 1, -1, 1, 2.20, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7024: [ 1, -1, 1, 2.41, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7027: [ 1, -1, 1, 2.39, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7029: [ 1, -1, 1, 2.38, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7032: [ 1, -1, 1, 2.40, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7038: [ 1, -1, 1, 2.42, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7043: [ 1, -1, 1, 2.41, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7046: [ 1, -1, 1, 2.43, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7049: [ 1, -1, 1, 2.45, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7069: [ 1, -1, 1, 2.43, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7072: [ 1, -1, 1, 2.45, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7076: [ 1, -1, 1, 2.43, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7081: [ 1, -1, 1, 2.40, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7084: [ 1, -1, 1, 2.44, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7089: [ 1, -1, 1, 2.40, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7099: [ 1, -1, 1, 2.42, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7209: [ 1, -1, 1, 2.43, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7222: [ 1, -1, 1, 2.45, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7231: [ 1, -1, 1, 2.45, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7235: [ 1, -1, 1, 2.45, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7247: [ 1, -1, 1, 2.46, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7267: [ 1, -1, 1, 2.45, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7269: [ 1, -1, 1, 2.45, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7284: [ 1, -1, 1, 2.43, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7389: [ 1, -1, 1, 2.51, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7419: [ 1, -1, 1, 2.48, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7423: [ 1, -1, 1, 2.48, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7424: [ 1, -1, 1, 2.53, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7426: [ 1, -1, 1, 2.46, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7428: [ 1, -1, 1, 2.49, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7431: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7436: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7444: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7475: [ 1, -1, 1, 2.52, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7549: [ 1, -1, 1, 2.52, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7584: [ 1, -1, 1, 2.54, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7665: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7666: [ 1, -1, 1, 2.48, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7831: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7836: [ 1, -1, 1, 2.55, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7853: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7865: [ 1, -1, 1, 2.48, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7885: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7888: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7912: [ 1, -1, 1, 2.52, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7950: [ 1, -1, 1, 2.52, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7972: [ 1, -1, 1, 2.48, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7980: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7995: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 8007: [ 1, -1, 1, 2.52, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 8015: [ 1, -1, 1, 2.46, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 8055: [ 1, -1, 1, 2.53, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 8078: [ 1, -1, 1, 9.00, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/iasi_metop-b.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/iasi_metop-b.yaml new file mode 100644 index 000000000..fc255a915 --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/atmosphere/iasi_metop-b.yaml @@ -0,0 +1,637 @@ +# Instrument metadata +# ------------------- +commissioned: 2013-07-31T18:00:00 + +observer_type: satellite # Type of chronicle to use + +# Instrument initial configuration +# -------------------------------- +channel_variables: + simulated: min + active: min + biascorrtd: min + error: max + error_cld: max + ermax: max + var_b: min + var_pg: min + icld_det: min + icloud: min + iaerosol: min +channel_values: + 16: [ 1, 1, 1, 1.38, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 29: [ 1, -1, 1, 0.81, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 32: [ 1, -1, 1, 0.75, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 35: [ 1, -1, 1, 0.79, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 38: [ 1, 1, 1, 0.72, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 41: [ 1, -1, 1, 0.74, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 44: [ 1, -1, 1, 0.68, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 47: [ 1, -1, 1, 0.72, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 49: [ 1, 1, 1, 0.65, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 50: [ 1, -1, 1, 0.65, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 51: [ 1, 1, 1, 0.65, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 53: [ 1, -1, 1, 0.69, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 55: [ 1, 1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 56: [ 1, -1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 57: [ 1, 1, 1, 0.65, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 59: [ 1, 1, 1, 0.67, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 61: [ 1, 1, 1, 0.62, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 62: [ 1, -1, 1, 0.61, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 63: [ 1, 1, 1, 0.62, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 66: [ 1, 1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 68: [ 1, -1, 1, 0.59, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 70: [ 1, 1, 1, 0.76, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 72: [ 1, 1, 1, 1.22, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 74: [ 1, 1, 1, 0.78, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 76: [ 1, -1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 78: [ 1, -1, 1, 0.62, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 79: [ 1, 1, 1, 0.61, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 81: [ 1, 1, 1, 0.69, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 82: [ 1, -1, 1, 0.65, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 83: [ 1, 1, 1, 0.59, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 84: [ 1, -1, 1, 0.61, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 85: [ 1, 1, 1, 0.59, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 86: [ 1, -1, 1, 0.68, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 87: [ 1, 1, 1, 0.62, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 89: [ 1, -1, 1, 0.68, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 92: [ 1, -1, 1, 4.38, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 93: [ 1, -1, 1, 3.05, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 95: [ 1, -1, 1, 2.31, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 97: [ 1, -1, 1, 1.56, 0.00, 4.00, 10.00, 0.00, -1, -1, -1 ] + 99: [ 1, -1, 1, 1.33, 0.00, 4.00, 10.00, 0.00, -1, -1, -1 ] + 101: [ 1, -1, 1, 1.58, 0.00, 3.50, 10.00, 0.00, -1, -1, -1 ] + 103: [ 1, -1, 1, 0.93, 0.00, 2.50, 10.00, 0.00, -1, -1, -1 ] + 104: [ 1, 1, 1, 1.67, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 106: [ 1, 1, 1, 0.72, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 109: [ 1, 1, 1, 0.57, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 110: [ 1, -1, 1, 0.58, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 111: [ 1, 1, 1, 0.55, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 113: [ 1, 1, 1, 0.68, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 116: [ 1, 1, 1, 0.59, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 119: [ 1, 1, 1, 0.68, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 122: [ 1, 1, 1, 0.59, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 125: [ 1, 1, 1, 0.65, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 128: [ 1, 1, 1, 0.58, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 131: [ 1, 1, 1, 0.62, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 133: [ 1, 1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 135: [ 1, 1, 1, 0.58, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 138: [ 1, 1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 141: [ 1, 1, 1, 0.55, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 144: [ 1, 1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 146: [ 1, 1, 1, 0.50, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 148: [ 1, 1, 1, 0.82, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 150: [ 1, -1, 1, 0.59, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 151: [ 1, 1, 1, 0.62, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 154: [ 1, 1, 1, 0.51, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 157: [ 1, 1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 159: [ 1, 1, 1, 0.52, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 160: [ 1, -1, 1, 0.51, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 161: [ 1, 1, 1, 0.51, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 163: [ 1, 1, 1, 0.76, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 167: [ 1, 1, 1, 0.52, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 170: [ 1, 1, 1, 0.57, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 173: [ 1, 1, 1, 0.55, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 176: [ 1, 1, 1, 0.69, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 179: [ 1, -1, 1, 0.58, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 180: [ 1, 1, 1, 0.65, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 185: [ 1, 1, 1, 0.61, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 187: [ 1, 1, 1, 0.59, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 191: [ 1, -1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 193: [ 1, 1, 1, 0.76, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 197: [ 1, -1, 1, 0.72, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 199: [ 1, 1, 1, 1.05, 0.00, 3.50, 10.00, 0.00, 1, -1, -1 ] + 200: [ 1, -1, 1, 0.75, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 202: [ 1, -1, 1, 0.51, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 203: [ 1, -1, 1, 0.65, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 205: [ 1, 1, 1, 1.30, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 207: [ 1, 1, 1, 0.69, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 210: [ 1, 1, 1, 0.93, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 212: [ 1, 1, 1, 1.49, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 213: [ 1, -1, 1, 1.12, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 214: [ 1, 1, 1, 0.68, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 217: [ 1, 1, 1, 0.66, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 218: [ 1, -1, 1, 0.67, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 219: [ 1, 1, 1, 0.59, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 222: [ 1, 1, 1, 0.59, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 224: [ 1, 1, 1, 0.69, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 225: [ 1, -1, 1, 0.67, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 226: [ 1, 1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 228: [ 1, -1, 1, 0.62, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 230: [ 1, 1, 1, 0.72, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 231: [ 1, -1, 1, 0.69, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 232: [ 1, 1, 1, 0.66, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 236: [ 1, 1, 1, 0.79, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 237: [ 1, -1, 1, 0.78, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 239: [ 1, 1, 1, 0.74, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 243: [ 1, 1, 1, 0.88, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 246: [ 1, 1, 1, 0.77, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 249: [ 1, 1, 1, 0.88, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 252: [ 1, 1, 1, 0.86, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 254: [ 1, 1, 1, 1.00, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 259: [ 1, -1, 1, 0.87, 0.00, 2.50, 10.00, 0.00, -1, -1, -1 ] + 260: [ 1, 1, 1, 0.85, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 262: [ 1, 1, 1, 0.88, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 265: [ 1, 1, 1, 0.84, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 267: [ 1, 1, 1, 0.84, 0.00, 3.50, 10.00, 0.00, 1, -1, -1 ] + 269: [ 1, -1, 1, 0.84, 0.00, 2.50, 10.00, 0.00, -1, -1, -1 ] + 275: [ 1, 1, 1, 0.80, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 279: [ 1, -1, 1, 0.80, 0.00, 3.00, 10.00, 0.00, -1, -1, -1 ] + 282: [ 1, 1, 1, 0.87, 0.00, 3.50, 10.00, 0.00, 1, -1, -1 ] + 285: [ 1, -1, 1, 0.98, 0.00, 3.00, 10.00, 0.00, -1, -1, -1 ] + 294: [ 1, 1, 1, 0.52, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 296: [ 1, 1, 1, 0.65, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 299: [ 1, 1, 1, 0.69, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 300: [ 1, -1, 1, 0.61, 0.00, 4.00, 10.00, 0.00, -1, -1, -1 ] + 303: [ 1, 1, 1, 0.60, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 306: [ 1, 1, 1, 0.67, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 309: [ 1, -1, 1, 0.79, 0.00, 4.50, 10.00, 0.00, -1, -1, -1 ] + 313: [ 1, -1, 1, 0.62, 0.00, 4.50, 10.00, 0.00, -1, -1, -1 ] + 320: [ 1, -1, 1, 0.66, 0.00, 4.50, 10.00, 0.00, -1, -1, -1 ] + 323: [ 1, 1, 1, 0.70, 0.00, 4.50, 10.00, 0.00, 1, -1, -1 ] + 326: [ 1, -1, 1, 0.65, 0.00, 4.50, 10.00, 0.00, -1, -1, -1 ] + 327: [ 1, 1, 1, 0.62, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 329: [ 1, 1, 1, 0.61, 0.00, 4.50, 10.00, 0.00, 1, -1, -1 ] + 332: [ 1, -1, 1, 0.62, 0.00, 4.00, 10.00, 0.00, -1, -1, -1 ] + 335: [ 1, 1, 1, 0.53, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 345: [ 1, 1, 1, 0.60, 0.00, 4.50, 10.00, 0.00, 1, -1, -1 ] + 347: [ 1, 1, 1, 0.68, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 350: [ 1, 1, 1, 0.95, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 354: [ 1, 1, 1, 0.63, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 356: [ 1, 1, 1, 0.97, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 360: [ 1, 1, 1, 0.65, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 363: [ 1, -1, 1, 0.98, 0.00, 3.00, 10.00, 0.00, -1, -1, -1 ] + 366: [ 1, 1, 1, 0.58, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 371: [ 1, 1, 1, 0.73, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 372: [ 1, -1, 1, 0.65, 0.00, 2.50, 10.00, 0.00, -1, -1, -1 ] + 373: [ 1, 1, 1, 0.85, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 375: [ 1, 1, 1, 0.99, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 377: [ 1, 1, 1, 0.76, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 379: [ 1, 1, 1, 0.85, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 381: [ 1, 1, 1, 0.97, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 383: [ 1, 1, 1, 0.77, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 386: [ 1, 1, 1, 0.62, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 389: [ 1, 1, 1, 0.63, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 398: [ 1, 1, 1, 1.21, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 401: [ 1, 1, 1, 1.41, 0.00, 4.50, 10.00, 0.00, 1, -1, -1 ] + 404: [ 1, 1, 1, 1.55, 0.00, 4.50, 10.00, 0.00, 1, -1, -1 ] + 405: [ 1, -1, 1, 1.78, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 407: [ 1, 1, 1, 1.35, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 408: [ 1, -1, 1, 1.14, 0.00, 4.00, 10.00, 0.00, -1, -1, -1 ] + 410: [ 1, 1, 1, 1.69, 0.00, 5.00, 10.00, 0.00, 1, -1, -1 ] + 411: [ 1, -1, 1, 1.79, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 414: [ 1, 1, 1, 1.46, 0.00, 5.00, 10.00, 0.00, 1, -1, -1 ] + 416: [ 1, 1, 1, 1.63, 0.00, 5.00, 10.00, 0.00, 1, -1, -1 ] + 418: [ 1, -1, 1, 1.94, 0.00, 5.50, 10.00, 0.00, -1, -1, -1 ] + 423: [ 1, -1, 1, 2.01, 0.00, 5.50, 10.00, 0.00, -1, -1, -1 ] + 426: [ 1, 1, 1, 1.24, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 428: [ 1, 1, 1, 1.76, 0.00, 5.00, 10.00, 0.00, 1, -1, -1 ] + 432: [ 1, 1, 1, 1.26, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 433: [ 1, -1, 1, 1.47, 0.00, 4.50, 10.00, 0.00, -1, -1, -1 ] + 434: [ 1, 1, 1, 1.90, 0.00, 5.50, 10.00, 0.00, 1, -1, -1 ] + 439: [ 1, 1, 1, 1.66, 0.00, 5.50, 10.00, 0.00, 1, -1, -1 ] + 442: [ 1, -1, 1, 2.13, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 445: [ 1, 1, 1, 1.49, 0.00, 4.50, 10.00, 0.00, 1, -1, -1 ] + 450: [ 1, -1, 1, 1.52, 0.00, 4.50, 10.00, 0.00, -1, -1, -1 ] + 457: [ 1, 1, 1, 1.55, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 459: [ 1, -1, 1, 1.96, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 472: [ 1, -1, 1, 2.31, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 477: [ 1, -1, 1, 2.33, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 483: [ 1, -1, 1, 2.32, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 509: [ 1, -1, 1, 2.31, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 515: [ 1, 1, 1, 2.33, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 546: [ 1, 1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 552: [ 1, 1, 1, 2.33, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 559: [ 1, 1, 1, 1.84, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 566: [ 1, 1, 1, 2.29, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 571: [ 1, 1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 573: [ 1, 1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 578: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 584: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 594: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 625: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 646: [ 1, 1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 662: [ 1, 1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 668: [ 1, 1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 705: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 739: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 756: [ 1, 1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 797: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 867: [ 1, 1, 1, 2.30, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 906: [ 1, 1, 1, 2.15, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 921: [ 1, 1, 1, 2.31, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1027: [ 1, 1, 1, 2.37, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1046: [ 1, 1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1090: [ 1, -1, 1, 2.29, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1098: [ 1, -1, 1, 2.29, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1121: [ 1, 1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1133: [ 1, 1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1173: [ 1, -1, 1, 2.32, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1191: [ 1, 1, 1, 2.32, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1194: [ 1, 1, 1, 2.31, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1222: [ 1, -1, 1, 2.32, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1271: [ 1, 1, 1, 2.32, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1283: [ 1, -1, 1, 2.31, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1338: [ 1, -1, 1, 2.31, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1409: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1414: [ 1, -1, 1, 2.29, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1420: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1424: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1427: [ 1, -1, 1, 2.29, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1430: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1434: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1440: [ 1, -1, 1, 2.25, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1442: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1445: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1450: [ 1, -1, 1, 2.21, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1454: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1460: [ 1, -1, 1, 2.17, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1463: [ 1, -1, 1, 2.18, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1469: [ 1, -1, 1, 2.17, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1474: [ 1, -1, 1, 2.21, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1479: [ 1, 1, 1, 1.99, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1483: [ 1, -1, 1, 2.16, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1487: [ 1, -1, 1, 2.20, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1494: [ 1, -1, 1, 2.13, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1496: [ 1, -1, 1, 2.12, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1502: [ 1, -1, 1, 2.13, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1505: [ 1, -1, 1, 2.10, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1509: [ 1, 1, 1, 2.12, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1510: [ 1, -1, 1, 2.11, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1513: [ 1, 1, 1, 2.09, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1518: [ 1, -1, 1, 2.09, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1521: [ 1, 1, 1, 2.08, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1526: [ 1, -1, 1, 2.09, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1529: [ 1, -1, 1, 2.04, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1532: [ 1, -1, 1, 2.04, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1536: [ 1, 1, 1, 2.10, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1537: [ 1, -1, 1, 2.01, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1541: [ 1, -1, 1, 2.05, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1545: [ 1, -1, 1, 2.03, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1548: [ 1, -1, 1, 2.06, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1553: [ 1, -1, 1, 1.98, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1560: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1568: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1574: [ 1, 1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1579: [ 1, 1, 1, 1.70, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1583: [ 1, -1, 1, 1.76, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1585: [ 1, 1, 1, 1.77, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1587: [ 1, 1, 1, 1.83, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1606: [ 1, -1, 1, 2.04, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1626: [ 1, 1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1639: [ 1, 1, 1, 1.99, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1643: [ 1, 1, 1, 1.99, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1652: [ 1, 1, 1, 2.07, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1658: [ 1, 1, 1, 2.02, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1659: [ 1, -1, 1, 2.04, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1666: [ 1, -1, 1, 2.10, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1671: [ 1, 1, 1, 2.06, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1675: [ 1, -1, 1, 2.18, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1681: [ 1, -1, 1, 2.21, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1694: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1697: [ 1, -1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1710: [ 1, -1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1786: [ 1, 1, 1, 1.98, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1791: [ 1, -1, 1, 2.20, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1805: [ 1, 1, 1, 2.18, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1839: [ 1, -1, 1, 2.18, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1884: [ 1, 1, 1, 2.21, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1913: [ 1, -1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1946: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1947: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1991: [ 1, 1, 1, 2.25, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2019: [ 1, 1, 1, 1.80, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2094: [ 1, 1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2119: [ 1, 1, 1, 1.73, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2213: [ 1, 1, 1, 1.73, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2239: [ 1, 1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2271: [ 1, 1, 1, 1.67, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2289: [ 1, -1, 1, 2.21, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2321: [ 1, 1, 1, 1.72, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2333: [ 1, -1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2346: [ 1, -1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2349: [ 1, -1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2352: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2359: [ 1, -1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2367: [ 1, -1, 1, 2.12, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2374: [ 1, -1, 1, 2.17, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2398: [ 1, 1, 1, 1.74, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2426: [ 1, -1, 1, 2.02, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2562: [ 1, -1, 1, 1.88, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2701: [ 1, 1, 1, 1.67, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2741: [ 1, -1, 1, 1.73, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2745: [ 1, -1, 1, 1.83, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2760: [ 1, -1, 1, 1.82, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2819: [ 1, -1, 1, 1.73, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2889: [ 1, 1, 1, 1.83, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2907: [ 1, -1, 1, 2.19, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2910: [ 1, -1, 1, 1.84, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2919: [ 1, -1, 1, 1.89, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2921: [ 1, -1, 1, 1.60, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2939: [ 1, -1, 1, 1.71, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2944: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2945: [ 1, -1, 1, 1.85, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2948: [ 1, -1, 1, 1.84, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2951: [ 1, -1, 1, 1.87, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2958: [ 1, 1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2971: [ 1, -1, 1, 1.52, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2977: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2985: [ 1, -1, 1, 1.87, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2988: [ 1, -1, 1, 1.89, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2990: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2991: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2993: [ 1, 1, 1, 1.93, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3002: [ 1, 1, 1, 1.90, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3008: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3014: [ 1, -1, 1, 1.90, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3027: [ 1, -1, 1, 1.89, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3029: [ 1, -1, 1, 1.89, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3030: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3036: [ 1, -1, 1, 1.90, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3047: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3049: [ 1, 1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3052: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3053: [ 1, -1, 1, 1.93, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3055: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3058: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3064: [ 1, -1, 1, 1.92, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3069: [ 1, -1, 1, 1.77, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3087: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3093: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3098: [ 1, -1, 1, 1.19, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3105: [ 1, 1, 1, 1.96, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3107: [ 1, -1, 1, 1.98, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3110: [ 1, 1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3116: [ 1, -1, 1, 1.55, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3127: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3129: [ 1, -1, 1, 1.92, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3136: [ 1, -1, 1, 1.92, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3146: [ 1, -1, 1, 1.97, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3151: [ 1, -1, 1, 1.93, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3160: [ 1, -1, 1, 1.99, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3165: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3168: [ 1, -1, 1, 1.12, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3175: [ 1, -1, 1, 1.93, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3178: [ 1, -1, 1, 1.92, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3189: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3207: [ 1, -1, 1, 1.85, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3228: [ 1, -1, 1, 1.84, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3244: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3248: [ 1, -1, 1, 1.12, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3252: [ 1, -1, 1, 1.82, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3256: [ 1, -1, 1, 1.82, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3263: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3281: [ 1, -1, 1, 1.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3295: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3303: [ 1, -1, 1, 1.96, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3309: [ 1, -1, 1, 1.21, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3312: [ 1, -1, 1, 1.83, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3322: [ 1, -1, 1, 1.96, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3326: [ 1, -1, 1, 1.36, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3354: [ 1, -1, 1, 1.96, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3366: [ 1, -1, 1, 1.82, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3375: [ 1, -1, 1, 1.92, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3378: [ 1, -1, 1, 1.68, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3411: [ 1, -1, 1, 1.93, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3416: [ 1, -1, 1, 1.23, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3432: [ 1, -1, 1, 1.96, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3438: [ 1, -1, 1, 1.93, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3440: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3442: [ 1, -1, 1, 1.41, 0.00, 5.50, 10.00, 0.00, -1, -1, -1 ] + 3444: [ 1, -1, 1, 1.16, 0.00, 4.50, 10.00, 0.00, -1, -1, -1 ] + 3446: [ 1, -1, 1, 1.60, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3448: [ 1, -1, 1, 1.25, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 3450: [ 1, -1, 1, 1.20, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 3452: [ 1, -1, 1, 1.65, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3454: [ 1, -1, 1, 1.66, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3458: [ 1, -1, 1, 1.87, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3467: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3476: [ 1, -1, 1, 1.96, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3484: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3491: [ 1, -1, 1, 1.25, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 3497: [ 1, -1, 1, 1.93, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3499: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3504: [ 1, -1, 1, 1.70, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3506: [ 1, -1, 1, 0.99, 0.00, 4.00, 10.00, 0.00, -1, -1, -1 ] + 3509: [ 1, -1, 1, 1.81, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3518: [ 1, -1, 1, 1.92, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3527: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3555: [ 1, -1, 1, 1.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3575: [ 1, -1, 1, 1.47, 0.00, 4.50, 10.00, 0.00, -1, -1, -1 ] + 3577: [ 1, -1, 1, 1.15, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3580: [ 1, -1, 1, 1.58, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3582: [ 1, -1, 1, 1.18, 0.00, 4.50, 10.00, 0.00, -1, -1, -1 ] + 3586: [ 1, -1, 1, 1.82, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3589: [ 1, -1, 1, 1.13, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3599: [ 1, -1, 1, 1.83, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3610: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3626: [ 1, -1, 1, 1.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3638: [ 1, -1, 1, 1.27, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 3646: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3653: [ 1, -1, 1, 1.45, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3658: [ 1, -1, 1, 1.60, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3661: [ 1, -1, 1, 1.29, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 3673: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3689: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3700: [ 1, -1, 1, 1.23, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 3710: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3726: [ 1, -1, 1, 1.21, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 3763: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3814: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3841: [ 1, -1, 1, 1.90, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3888: [ 1, -1, 1, 1.33, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 4032: [ 1, -1, 1, 1.75, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4059: [ 1, -1, 1, 2.02, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4068: [ 1, -1, 1, 1.98, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4082: [ 1, -1, 1, 2.03, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4095: [ 1, -1, 1, 1.83, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4160: [ 1, -1, 1, 1.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4234: [ 1, -1, 1, 2.04, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4257: [ 1, -1, 1, 2.02, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4411: [ 1, -1, 1, 1.90, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4498: [ 1, -1, 1, 2.00, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4520: [ 1, -1, 1, 2.02, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4552: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4567: [ 1, -1, 1, 1.93, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4608: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4646: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4698: [ 1, -1, 1, 1.99, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4808: [ 1, -1, 1, 2.00, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4849: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4920: [ 1, -1, 1, 1.96, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4939: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4947: [ 1, -1, 1, 1.92, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4967: [ 1, -1, 1, 1.88, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4991: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4996: [ 1, -1, 1, 1.84, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5015: [ 1, -1, 1, 1.87, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5028: [ 1, -1, 1, 1.77, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5056: [ 1, -1, 1, 1.89, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5128: [ 1, -1, 1, 1.89, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5130: [ 1, -1, 1, 1.88, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5144: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5170: [ 1, -1, 1, 1.82, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5178: [ 1, -1, 1, 1.79, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5183: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5188: [ 1, -1, 1, 2.06, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5191: [ 1, -1, 1, 2.33, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5368: [ 1, -1, 1, 1.88, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5371: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5379: [ 1, -1, 1, 1.81, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5381: [ 1, 1, 1, 1.80, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5383: [ 1, -1, 1, 1.80, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5397: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5399: [ 1, 1, 1, 1.90, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5401: [ 1, -1, 1, 2.00, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5403: [ 1, -1, 1, 2.06, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5405: [ 1, -1, 1, 2.10, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5446: [ 1, -1, 1, 2.20, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5455: [ 1, -1, 1, 2.00, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5472: [ 1, -1, 1, 2.16, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5480: [ 1, 1, 1, 1.98, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5483: [ 1, -1, 1, 1.80, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5485: [ 1, -1, 1, 1.80, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5492: [ 1, -1, 1, 1.85, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5497: [ 1, -1, 1, 1.75, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5502: [ 1, -1, 1, 2.04, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5507: [ 1, -1, 1, 2.19, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5509: [ 1, -1, 1, 2.14, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5517: [ 1, -1, 1, 2.19, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5528: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5558: [ 1, -1, 1, 2.10, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5697: [ 1, -1, 1, 2.11, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5714: [ 1, -1, 1, 2.18, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5749: [ 1, -1, 1, 2.03, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5766: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5785: [ 1, -1, 1, 2.19, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5798: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5799: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5801: [ 1, -1, 1, 2.21, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5817: [ 1, -1, 1, 2.21, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5833: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5834: [ 1, -1, 1, 2.33, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5836: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5849: [ 1, -1, 1, 2.21, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5851: [ 1, -1, 1, 2.12, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5852: [ 1, -1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5865: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5869: [ 1, -1, 1, 2.25, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5881: [ 1, -1, 1, 1.88, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5884: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5897: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5900: [ 1, -1, 1, 2.36, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5916: [ 1, -1, 1, 2.29, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5932: [ 1, -1, 1, 2.35, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5948: [ 1, -1, 1, 2.30, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5963: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5968: [ 1, -1, 1, 2.08, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5978: [ 1, -1, 1, 2.05, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5988: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5992: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5994: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5997: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6003: [ 1, -1, 1, 1.97, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6008: [ 1, -1, 1, 2.25, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6023: [ 1, -1, 1, 2.25, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6026: [ 1, -1, 1, 2.25, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6039: [ 1, -1, 1, 2.31, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6053: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6056: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6067: [ 1, -1, 1, 2.13, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6071: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6082: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6085: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6098: [ 1, -1, 1, 2.41, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6112: [ 1, -1, 1, 2.34, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6126: [ 1, -1, 1, 9.32, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6135: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6140: [ 1, -1, 1, 2.38, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6149: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6154: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6158: [ 1, -1, 1, 2.39, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6161: [ 1, -1, 1, 2.11, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6168: [ 1, -1, 1, 2.09, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6174: [ 1, -1, 1, 2.10, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6182: [ 1, -1, 1, 2.06, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6187: [ 1, -1, 1, 2.12, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6205: [ 1, -1, 1, 2.08, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6209: [ 1, -1, 1, 2.00, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6213: [ 1, -1, 1, 1.93, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6317: [ 1, -1, 1, 2.02, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6339: [ 1, -1, 1, 2.55, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6342: [ 1, -1, 1, 1.54, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6366: [ 1, -1, 1, 1.64, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6381: [ 1, -1, 1, 1.51, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6391: [ 1, -1, 1, 1.55, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6489: [ 1, -1, 1, 2.82, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6962: [ 1, -1, 1, 2.92, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6966: [ 1, -1, 1, 2.55, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6970: [ 1, -1, 1, 2.37, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6975: [ 1, -1, 1, 1.85, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6977: [ 1, -1, 1, 1.60, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6982: [ 1, -1, 1, 1.72, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6985: [ 1, -1, 1, 1.74, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6987: [ 1, -1, 1, 1.79, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6989: [ 1, -1, 1, 1.90, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6991: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6993: [ 1, -1, 1, 2.00, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6995: [ 1, -1, 1, 2.04, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6997: [ 1, -1, 1, 2.08, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6999: [ 1, -1, 1, 2.12, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7000: [ 1, -1, 1, 2.13, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7004: [ 1, -1, 1, 2.16, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7008: [ 1, -1, 1, 2.18, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7013: [ 1, -1, 1, 2.18, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7016: [ 1, -1, 1, 2.20, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7021: [ 1, -1, 1, 2.20, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7024: [ 1, -1, 1, 2.41, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7027: [ 1, -1, 1, 2.39, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7029: [ 1, -1, 1, 2.38, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7032: [ 1, -1, 1, 2.40, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7038: [ 1, -1, 1, 2.42, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7043: [ 1, -1, 1, 2.41, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7046: [ 1, -1, 1, 2.43, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7049: [ 1, -1, 1, 2.45, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7069: [ 1, -1, 1, 2.43, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7072: [ 1, -1, 1, 2.45, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7076: [ 1, -1, 1, 2.43, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7081: [ 1, -1, 1, 2.40, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7084: [ 1, -1, 1, 2.44, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7089: [ 1, -1, 1, 2.40, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7099: [ 1, -1, 1, 2.42, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7209: [ 1, -1, 1, 2.43, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7222: [ 1, -1, 1, 2.45, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7231: [ 1, -1, 1, 2.45, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7235: [ 1, -1, 1, 2.45, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7247: [ 1, -1, 1, 2.46, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7267: [ 1, -1, 1, 2.45, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7269: [ 1, -1, 1, 2.45, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7284: [ 1, -1, 1, 2.43, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7389: [ 1, -1, 1, 2.51, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7419: [ 1, -1, 1, 2.48, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7423: [ 1, -1, 1, 2.48, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7424: [ 1, -1, 1, 2.53, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7426: [ 1, -1, 1, 2.46, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7428: [ 1, -1, 1, 2.49, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7431: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7436: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7444: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7475: [ 1, -1, 1, 2.52, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7549: [ 1, -1, 1, 2.52, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7584: [ 1, -1, 1, 2.54, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7665: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7666: [ 1, -1, 1, 2.48, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7831: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7836: [ 1, -1, 1, 2.55, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7853: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7865: [ 1, -1, 1, 2.48, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7885: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7888: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7912: [ 1, -1, 1, 2.52, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7950: [ 1, -1, 1, 2.52, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7972: [ 1, -1, 1, 2.48, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7980: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7995: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 8007: [ 1, -1, 1, 2.52, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 8015: [ 1, -1, 1, 2.46, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 8055: [ 1, -1, 1, 2.53, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 8078: [ 1, -1, 1, 9.00, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/ssmis_f17.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/ssmis_f17.yaml new file mode 100644 index 000000000..1d4c17a3a --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/atmosphere/ssmis_f17.yaml @@ -0,0 +1,45 @@ +# Instrument metadata +# ------------------- +commissioned: 2006-12-12T00:00:00 + +observer_type: satellite # Type of chronicle to use + +# Instrument initial configuration +# -------------------------------- +channel_variables: + simulated: min + active: min + biascorrtd: min + error: max + error_cld: max + ermax: max + var_b: min + var_pg: min + icld_det: min + icloud: min + iaerosol: min +channel_values: + 1: [ 1, 1, 1, 1.50, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 2: [ 1, -1, 1, 0.50, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 3: [ 1, -1, 1, 0.50, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 4: [ 1, -1, 1, 0.50, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 5: [ 1, 1, 1, 0.50, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 6: [ 1, 1, 1, 1.00, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 7: [ 1, 1, 1, 1.00, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 8: [ 1, -1, 1, 3.00, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 9: [ 1, -1, 1, 3.00, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 10: [ 1, -1, 1, 3.00, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 11: [ 1, -1, 1, 3.00, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 12: [ 1, -1, 1, 2.40, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 13: [ 1, -1, 1, 1.27, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 14: [ 1, -1, 1, 1.44, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 15: [ 1, -1, 1, 3.00, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 16: [ 1, -1, 1, 1.34, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 17: [ 1, -1, 1, 1.74, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 18: [ 1, -1, 1, 3.75, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 19: [ 1, -1, 1, 3.00, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 20: [ 1, -1, 1, 3.00, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 21: [ 1, -1, 1, 2.00, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 22: [ 1, -1, 1, 6.40, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 23: [ 1, -1, 1, 1.00, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 24: [ 1, 1, 1, 1.00, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/ssmis_f18.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/ssmis_f18.yaml new file mode 100644 index 000000000..9babb6114 --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/atmosphere/ssmis_f18.yaml @@ -0,0 +1,45 @@ +# Instrument metadata +# ------------------- +commissioned: 2009-12-15T00:00:00 + +observer_type: satellite # Type of chronicle to use + +# Instrument initial configuration +# -------------------------------- +channel_variables: + simulated: min + active: min + biascorrtd: min + error: max + error_cld: max + ermax: max + var_b: min + var_pg: min + icld_det: min + icloud: min + iaerosol: min +channel_values: + 1: [ 1, -1, 1, 1.50, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 2: [ 1, -1, 1, 0.50, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 3: [ 1, -1, 1, 0.50, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 4: [ 1, -1, 1, 0.50, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 5: [ 1, -1, 1, 0.50, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 6: [ 1, -1, 1, 1.00, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 7: [ 1, -1, 1, 1.00, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 8: [ 1, -1, 1, 3.00, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 9: [ 1, -1, 1, 3.00, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 10: [ 1, -1, 1, 3.00, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 11: [ 1, -1, 1, 3.00, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 12: [ 1, -1, 1, 2.40, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 13: [ 1, -1, 1, 1.27, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 14: [ 1, -1, 1, 1.44, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 15: [ 1, -1, 1, 3.00, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 16: [ 1, -1, 1, 1.34, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 17: [ 1, -1, 1, 1.74, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 18: [ 1, -1, 1, 3.75, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 19: [ 1, -1, 1, 3.00, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 20: [ 1, -1, 1, 3.00, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 21: [ 1, -1, 1, 2.00, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 22: [ 1, -1, 1, 6.40, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 23: [ 1, -1, 1, 1.00, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] + 24: [ 1, -1, 1, 1.00, 0.00, 6.00, 10.00, 0.00, -2, -1, -1 ] diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml.j2 deleted file mode 120000 index 9685f0007..000000000 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml.j2 +++ /dev/null @@ -1 +0,0 @@ -../atmosphere/aircraft.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 index 944e3e956..48c601276 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 @@ -1,5 +1,9 @@ -- obs space: - name: AMSU-A NOAA-19 +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: AMSUA N19 distribution: name: Halo halo size: 1250e3 @@ -14,18 +18,27 @@ io pool: max pool size: 1 simulated variables: [brightnessTemperature] - channels: &{{observation_from_jcb}}_channels {{ get_satellite_simulated_channels(observation_from_jcb) }} + channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_simulated_channels(observation_from_jcb) }} + + # Observation Operator + # -------------------- obs operator: name: CRTM - Absorbers: [H2O,O3] + Absorbers: [H2O, O3] Clouds: [Water, Ice] Cloud_Fraction: 1.0 obs options: - Sensor_ID: {{observation_from_jcb}} + Sensor_ID: &{{observation_from_jcb}}_sensor_id amsua_n19 EndianType: little_endian - CoefficientPath: ./crtm/ + CoefficientPath: "{{crtm_coefficient_path}}" + + # Observation Error + # ----------------- obs error: covariance model: diagonal + + # Observation Bias Correction (VarBC) + # ----------------------------------- obs bias: input file: "{{atm_obsbiasin_path}}/{{atm_obsbiasin_prefix}}{{observation_from_jcb}}{{atm_obsbiasin_suffix}}" output file: "{{atm_obsbiasout_path}}/{{atm_obsbiasout_prefix}}{{observation_from_jcb}}{{atm_obsbiasout_suffix}}" @@ -56,11 +69,14 @@ ratio: 1.1 ratio for small dataset: 2.0 output file: "{{atm_obsbiasout_path}}/{{atm_obsbiascovout_prefix}}{{observation_from_jcb}}{{atm_obsbiascovout_suffix}}" + + # Observation Filters (QC) + # ------------------------ obs filters: - filter: Bounds Check filter variables: - name: brightnessTemperature - channels: *{{observation_from_jcb}}_channels + channels: *{{observation_from_jcb}}_simulated_channels minvalue: 100.0 maxvalue: 500.0 action: @@ -69,11 +85,24 @@ - filter: Background Check filter variables: - name: brightnessTemperature - channels: *{{observation_from_jcb}}_channels + channels: *{{observation_from_jcb}}_simulated_channels threshold: 3.0 action: name: reject + + # Observation Localizations (LocalEnsembleDA) + # ------------------------------------------- obs localizations: - localization method: Horizontal Gaspari-Cohn lengthscale: 1250e3 max nobs: 10000 + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/conv_ps.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/conv_ps.yaml.j2 new file mode 100644 index 000000000..014fa5269 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/conv_ps.yaml.j2 @@ -0,0 +1,373 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: surface_ps + distribution: + name: Halo + halo size: 1250e3 + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [stationPressure] + + # Observation Operator + # -------------------- + obs operator: + name: SfcPCorrected + variables: + - name: stationPressure + da_psfc_scheme: GSI + station_altitude: height + geovar_sfc_geomz: surface_altitude + geovar_geomz: geopotential_height + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: Identity + variables: + - name: stationPressure + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Initial Error Assignments for SFC Observations + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [181] + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + round_to_the_nearest_integer: true + xvar: + name: ObsValue/stationPressure + xvals: [80000, 75000, 70000, 65000, 60000, 55000] + errors: [110, 120, 120, 120, 120, 1.0e+11] + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [187] + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + round_to_the_nearest_integer: true + xvar: + name: ObsValue/stationPressure + xvals: [85000, 80000, 75000, 70000, 65000, 60000, 55000] + errors: [120, 140, 140, 140, 140, 140, 1.0e+11] + # Initial Error Assignments for SFCSHIP Observations + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [180] + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + round_to_the_nearest_integer: true + xvar: + name: ObsValue/stationPressure + xvals: [60000, 55000] + errors: [130, 1.0e+11] + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [183] + action: + name: assign error + error parameter: 1.0e+11 + + # Initial Error Assignments for Radiosonde + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [120] + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + round_to_the_nearest_integer: true + xvar: + name: ObsValue/stationPressure + xvals: [80000, 75000, 70000, 65000, 60000, 55000] + errors: [110, 120, 120, 120, 120, 1.0e+11] + # Create PreQC group variable (pqm in GSI read_prepbufr) + - filter: Variable Assignment + assignments: + - name: InputObsError/stationPressure + type: float + source variable: ObsErrorData/stationPressure + + # Set observation quality-realted variables + # Create PreQC group variable (pqm in GSI read_prepbufr) + - filter: Variable Assignment + assignments: + - name: PreQC/stationPressure + type: int + source variable: QualityMarker/stationPressure + + # Create PreUseFlag group variable (usage in GSI read_prepbufr) + # Initialize + - filter: Variable Assignment + assignments: + - name: PreUseFlag/stationPressure + type: int + source variable: PreQC/stationPressure + + - filter: Variable Assignment + where: + - variable: + name: PreUseFlag/stationPressure + is_in: 1-15 + assignments: + - name: PreUseFlag/stationPressure + value: 0 + # Re-assignment + - filter: Variable Assignment + where: + - variable: + name: ObsType/stationPressure + is_in: 183 + assignments: + - name: PreUseFlag/stationPressure + value: 100 + + - filter: Variable Assignment + where: + - variable: + name: ObsValue/stationPressure + is_defined: + - variable: + name: ObsValue/stationPressure + maxvalue: 50000.00 + where operator: and + assignments: + - name: PreUseFlag/stationPressure + value: 100 + + - filter: Variable Assignment + where: + - variable: + name: PreQC/stationPressure + is_in: 9, 12, 15 + assignments: + - name: PreUseFlag/stationPressure + value: 100 + + - filter: Variable Assignment + where: + - variable: + name: PreQC/stationPressure + is_in: 4-15 + assignments: + - name: PreUseFlag/stationPressure + value: 101 + + ## Observation range sanity check + #- filter: Bounds Check + # filter variables: + # - name: stationPressure + # minvalue: 37499.0 + # maxvalue: 106999.0 + # action: + # name: reject + ## Reject all ObsType 183 + #- filter: RejectList + # where: + # - variable: + # name: ObsType/stationPressure + # is_in: 183 + ## Reject surface pressure below 500 hPa + #- filter: Bounds Check + # filter variables: + # - name: stationPressure + # minvalue: 50000.00 + # action: + # name: reject + #- filter: RejectList + # where: + # - variable: + # name: PreQC/stationPressure + # is_in: 4-15 + # Inflate obs error based on obs type + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: PreQC/stationPressure + is_in: 3, 7 + action: + name: inflate error + inflation factor: 1.2 + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Calculate obs error inflation factors for duplicated observations at the same location + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorDuplicateCheck/stationPressure + type: float + function: + name: ObsFunction/ObsErrorFactorDuplicateCheck + options: + use_air_pressure: false + variable: stationPressure + + # Reduce effective observation error based on obs type and subtype + # In this case: reduce effective obs error for buoy + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: + name: ObsType/stationPressure + is_in: 180 + - variable: + name: ObsSubType/stationPressure + is_in: 0 + action: + name: inflate error + inflation factor: 0.7 + + # Calculate obs error inflation factors for large discrepancies between model and observations + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorSfcPressure/stationPressure + type: float + function: + name: ObsFunction/ObsErrorFactorSfcPressure + options: + geovar_sfc_geomz: surface_altitude + geovar_geomz: geopotential_height + station_altitude: height + + # Inflate surface pressure observation based on discrepancies between + # model and observations due to terrian + - filter: Perform Action + filter variables: + - name: stationPressure + action: + name: inflate error + inflation variable: + name: ObsErrorFactorSfcPressure/stationPressure + + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/Innovation + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsValue/stationPressure + - name: HofX/stationPressure + coefs: [1, -1] + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ObsErrorBoundSfcPressure1 + type: float + function: + name: ObsFunction/ObsErrorBoundConventional + options: + obsvar: stationPressure + obserr_bound_min: 100 + obserr_bound_max: 300 + obserr_bound_factor: 5.0 + + - filter: Background Check + filter variables: + - name: stationPressure + where: + - variable: PreQC/stationPressure + is_not_in: 3 + function absolute threshold: + - name: DerivedMetaData/ObsErrorBoundSfcPressure1 + action: + name: reject + + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ObsErrorBoundSfcPressure2 + type: float + function: + name: ObsFunction/ObsErrorBoundConventional + options: + obsvar: stationPressure + obserr_bound_min: 100 + obserr_bound_max: 300 + obserr_bound_factor: 3.5 + + - filter: Background Check + filter variables: + - name: stationPressure + where: + - variable: PreQC/stationPressure + is_in: 3 + function absolute threshold: + - name: DerivedMetaData/ObsErrorBoundSfcPressure2 + action: + name: reject + + # Inflate obs error based on duplicate check + - filter: Perform Action + filter variables: + - name: stationPressure + action: + name: inflate error + inflation variable: + name: ObsErrorFactorDuplicateCheck/stationPressure + + # Reject data based on PreUseFlag (usage in GSI) + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: PreUseFlag/stationPressure + is_not_in: 0, 1 + action: + name: reject + # End of Filters + + # Observation Localizations (LocalEnsembleDA) + # ------------------------------------------- + obs localizations: + - localization method: Horizontal Gaspari-Cohn + lengthscale: 1250e3 + max nobs: 10000 + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/gnssro.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/gnssro.yaml.j2 new file mode 100644 index 000000000..5075a201a --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/gnssro.yaml.j2 @@ -0,0 +1,167 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: gnssrobndnbam + distribution: + name: Halo + halo size: 1250e3 + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + # obsgrouping: + # group variables: [ "sequenceNumber" ] + # sort variable: "impactHeightRO" + # sort order: "ascending" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + simulated variables: [bendingAngle] + + # Observation Operator + # -------------------- + obs operator: + name: GnssroBndNBAM + obs options: + use_compress: 1 + sr_steps: 2 + vertlayer: full + super_ref_qc: NBAM + + # Observation Filters (QC) + # ------------------------ + obs filters: + # Apply gross check using pccf + # Step 0-A: Create Diagnostic Flags + # Diagnostic flag for pccf + - filter: Create Diagnostic Flags + filter variables: + - name: bendingAngle + flags: + - name: pccfCheckReject + initial value: false + force reinitialization: true + + # Diagnostic flag for qfro + - filter: Create Diagnostic Flags + filter variables: + - name: bendingAngle + flags: + - name: qfroCheckReject + initial value: false + force reinitialization: true + + # Step 0-B: pccf Check - good: 0.1-100, reject: 0 + - filter: Bounds Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/satelliteIdentifier + is_in: 265-269,750-755,44,786,820,825 + test variables: + - name: MetaData/pccf + minvalue: 0.1 + maxvalue: 100.1 + actions: + - name: set + flag: pccfCheckReject + - name: reject + + # Step 0-B: qfro Check - good: 0, reject: 1 + - filter: Bounds Check + - filter: RejectList + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/satelliteIdentifier + is_in: 3-5,421,440,821 + test variables: + - name: MetaData/qualityFlags + minvalue: -0.1 + maxvalue: 0.1 + actions: + - name: set + flag: qfroCheckReject + - name: reject + + #1. gpstop + - filter: Domain Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/impactHeightRO + minvalue: 0 + maxvalue: 55000.1 + action: + name: reject + #2. commgpstop + - filter: Bounds Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/satelliteIdentifier + is_in: 265,266,267,268,269 + test variables: + - name: MetaData/impactHeightRO + maxvalue: 45000.1 + action: + name: reject + #3. metop below 8 km + - filter: Bounds Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/satelliteIdentifier + is_in: 3-5 + test variables: + - name: MetaData/impactHeightRO + minvalue: 8000.1 + action: + name: reject + #4. assign obs error + - filter: ROobserror + filter variables: + - name: bendingAngle + errmodel: NBAM + #5. RONBAM cut off check + - filter: Background Check RONBAM + filter variables: + - name: bendingAngle + #6. Obs error inflate + - filter: Background Check RONBAM + filter variables: + - name: bendingAngle + action: + name: RONBAMErrInflate + #7. Background check + #- filter: Background Check + # filter variables: + # - name: bendingAngle + # threshold: 10 + # action: + # name: reject + + # Observation Localizations (LocalEnsembleDA) + # ------------------------------------------- + obs localizations: + - localization method: Horizontal Gaspari-Cohn + lengthscale: 1250e3 + max nobs: 10000 + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwind_abi_goes-16.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwind_abi_goes-16.yaml.j2 new file mode 100644 index 000000000..2c276038e --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwind_abi_goes-16.yaml.j2 @@ -0,0 +1,611 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: satwind_goes-16 + distribution: + name: Halo + halo size: 1250e3 + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + # Observation Operator + # -------------------- + obs operator: + name: VertInterp + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: VertInterp + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: false + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Assign the initial observation error, based on height/pressure + # Hard-wiring to prepobs_errtable.global by Type + # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML + # Type 240 (GOES SWIR): Assigned all dummy values in prepobs_errtable.global + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 0.] #Pressure (Pa) + errors: [1000000000., 1000000000.] + # Type 245 (GOES LWIR): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 245 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, + 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., + 14., 14., 14., 14., 14., 14.] + # Type 246 (GOES cloud-top WV): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 246 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, + 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., + 14., 14., 14., 14., 14., 14.] + # Type 247 (GOES clear-sky WV): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 247 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, + 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., + 14., 14., 14., 14., 14., 14.] + # Type 251 (GOES VIS): Assigned all dummy values + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 251 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 0.] #Pressure (Pa) + errors: [1000000000., 1000000000.] + # sanity-check criteria + # Observation Range Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130. + maxvalue: 130. + action: + name: reject + + # Velocity Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130. + action: + name: reject + + # GSI read routine QC (part-1) + # Exclude Type 251 (VIS) --- obs tossed without passing to setup routine + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 251 + action: + name: reject + + # Exclude data with satellite zenith angle > 68 for all types --- obs tossed without passing to setup routine + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: MetaData/satelliteZenithAngle + maxvalue: 68. + action: + name: reject + + # Reject obs with pressure < 12500 pa --- obs tosed without passing to setup routine + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + test variables: + - name: MetaData/pressure + minvalue: 12500. + action: + name: reject + + # Exclude data over non-water surface type where latitude > 20N for Type 240 (IRSW) and Type 245 (IRLW) --- obs tossed and not passed to setup routine + # Notes: This check was missing, so added (eliu) + # Replace land_type_index_NPOSS with water_area_fraction (eliu) + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240, 245 + - variable: MetaData/latitude + minvalue: 20. + test variables: + - name: GeoVaLs/water_area_fraction + minvalue: 0.99 + action: + name: reject + + # GSI read routine QC (part-2) + # Reject obs with qualityInformationWithoutForecast < 90. OR > 100. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + test variables: + - name: MetaData/qualityInformationWithoutForecast + minvalue: 90. + maxvalue: 100. + action: + name: reject + + # Reject obs with pressure < 15000 pa. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + test variables: + - name: MetaData/pressure + minvalue: 15000. + action: + name: reject + + # Reject obs with pressure < 70000 pa. when Type=251 + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + - variable: ObsType/windEastward + is_in: 251 + test variables: + - name: MetaData/pressure + minvalue: 70000. + action: + name: reject + + # Reject obs with pressure > 30000. when Type=246 + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + - variable: ObsType/windEastward + is_in: 246 + test variables: + - name: MetaData/pressure + maxvalue: 30000. + action: + name: reject + + # Reject obs with pressure > 85000. when isli=1 (land surface) + # Notes: Replace land_type_index_NPOESS with land_area_fraction (eliu) + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + - variable: GeoVaLs/land_area_fraction + minvalue: 0.99 + test variables: + - name: MetaData/pressure + maxvalue: 85000. + action: + name: reject + + # Reject obs with pct1 (Coeff. of Var.) outside of 0.04–0.5, Type [240,245,246,251] ONLY + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + - variable: ObsType/windEastward + is_in: 240, 245, 246, 251 + test variables: + - name: MetaData/coefficientOfVariation + minvalue: 0.04 + maxvalue: 0.5 + action: + name: reject + + # NESDIS obs are also subject to the experr_norm test defined as: + # + # if (10. - 0.1*(expectedError))/(ob_speed)>0.9, or ob_speed<0.1, reject, applies to NESDIS winds + # + # CLEARED: With caveat that float precision/handling differences can generate different acceptance criteria + # between UFO and GSI for observations with an experr_norm value right around the maxvalue. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + test variables: + - name: ObsFunction/SatWindsErrnormCheck + maxvalue: 0.9 + action: + name: reject + + # Reject all Type=240 (GOES SWIR) AMVs: These are not currently assimilated in GSI and they have missing-values + # assigned to ob-errors + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240 + action: + name: reject + + # GSI setupw routine QC + # Reject any ob Type [240–260] when pressure greater than 950 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240-260 + test variables: + - name: MetaData/pressure + maxvalue: 95001. + action: + name: reject + + # GOES IR (245) reject when pressure between 399 and 801 mb. + # # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/pressure + minvalue: 39901. + maxvalue: 80099. + - variable: ObsType/windEastward + is_in: 245 + action: + name: reject + + # GOES WV (246, 250, 254), reject when pressure greater than 399 mb. + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 246, 250, 254 + test variables: + - name: MetaData/pressure + maxvalue: 39900. + action: + name: reject + + # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation + # Turn this check off for now. + # Need to check if troposphere pressure was implemented correctly in fv3-jed + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + reference: GeoVaLs/tropopause_pressure + value: MetaData/pressure + minvalue: -5000. # 50 hPa above tropopause level, negative p-diff + action: + name: reject + + # GOES (247) reject any observation with a /=0 surface type (non-water + # surface) within 110 hPa of the surface pressure (as part of the LNVD # check). + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + # Notes (eliu): Replace land_type_index_NPOESS with land_area_fraction. + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + - variable: + name: ObsType/windEastward + is_in: 247 + reference: GeoVaLs/surface_pressure + value: MetaData/pressure + maxvalue: -11000. # within 110 hPa above surface pressure, negative p-diff + action: + name: reject + + # Reject GOES (247) when difference of wind direction is more than 50 degrees. + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 247 + test variables: + - name: ObsFunction/WindDirAngleDiff + maxvalue: 50. + action: + name: reject + + # All satwinds must adjust errors based on ObsErrorFactorPressureCheck + # prior to the SPDB check (i.e. the gross-error check). The gross-error + # check uses the adjusted errors for error-bound tightening and rejection, + # so this check has to come first. This check will inflate errors for obs + # that are too close to either the model top or bottom. + # Notes (eliu): GMAO added a required parameter: adjusted_error_name. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + surface_obs: false + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All satwinds subject to a gross-error check that contains significant + # modifiers for satwinds with a negative speed-bias. ALL wind gross-error + # checks are currently being done by the SatWindsSPDBCheck. + # CLEARED + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, + 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, + 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, + 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, + 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, + 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, + 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, + 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, + 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don"t want to inflate errors for duplication if one + # of the duplicates should be rejected. + # Notes (eliu): ObsErrorFactorDuplicateCheck obsfunction requires PreUseFlag (usage parameter from read_satwnd.f90). + # : Turn off duplicate check for now. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + # We are extending this to an additional filter that inflates final ob-errors across-the-board by + # 1/0.8 = 1.25. This is caused by the GSI value of nvqc being set to .true. in the global operational + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this nvqc functionality were to be switched off (i.e. if variational qc were to be turned off), + # you would want to remove this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + # End of Filters + + # Observation Localizations (LocalEnsembleDA) + # ------------------------------------------- + obs localizations: + - localization method: Horizontal Gaspari-Cohn + lengthscale: 1250e3 + max nobs: 10000 + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwind_ahi_h8.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwind_ahi_h8.yaml.j2 new file mode 100644 index 000000000..f4ce28c48 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwind_ahi_h8.yaml.j2 @@ -0,0 +1,442 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: satwind_ahi_h8 + distribution: + name: Halo + halo size: 1250e3 + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + # Observation Operator + # -------------------- + obs operator: + name: VertInterp + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: VertInterp + + # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI"s thinning of AHI/Himawari-8 satwinds + # results in more JEDI satwinds in the diag file than in GSI, but far fewer JEDI satwinds assimilated than + # GSI. JEDI under-counts assimilated winds by roughly 25-40%, relative to GSI, and this under-count is not + # even including the temporal thinning which is applied in GSI but not JEDI (by this filter below). See + # GDASApp Issue #741 for details: https://github.com/NOAA-EMC/GDASApp/issues/741 + #obs pre filters: + #- filter: Gaussian Thinning + # horizontal_mesh: 200 + # vertical_mesh: 10000 + # use_reduced_horizontal_grid: true + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: false + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Assign the initial observation error, based on height/pressure + # Hard-wiring to prepobs_errtable.global by Type + # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML + # Type 242 (Himawari VIS) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 242 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, + 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., + 7., 7., 7.] + # Type 250 (Himawari AHI WV, cloud-top or clear-sky) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 250 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, + 5., 7., 7.3, 7.6, 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., + 8., 8., 8.] + # Type Type 252 (Himawari AHI LWIR) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 252 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, + 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., + 7., 7., 7.] + # sanity-check criteria + # Observation Range Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130. + maxvalue: 130. + action: + name: reject + + # Velocity Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130. + action: + name: reject + + # GSI read routine QC (part-1) + # Exclude Type 250 with windComputationMethod==5 (clear-sky WV) --- obs tossed without passing to setup routine + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windNorthward + is_in: 250 + - variable: MetaData/windComputationMethod + is_in: 5 + action: + name: reject + + # Exclude data with satellite zenith angle > 68 for all types --- obs tossed without passing to setup routine + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: MetaData/satelliteZenithAngle + maxvalue: 68. + action: + name: reject + + # GSI read routine QC (part-2) + # Reject obs with qualityInformationWithoutForecast < 85. (also > 100., which is missing data) + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: MetaData/qualityInformationWithoutForecast + minvalue: 85. + maxvalue: 100. + action: + name: reject + + # Reject Type 252 (IR) winds with a /=0 surface type (non-water surface) when latitude > 20. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + - variable: + name: ObsType/windEastward + is_in: 252 + test variables: + - name: MetaData/latitude + maxvalue: 20. + action: + name: reject + + # GSI setupw routine QC + # Reject any ob Type [240–260] when pressure greater than 950 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240-260 + test variables: + - name: MetaData/pressure + maxvalue: 95001. + action: + name: reject + + # IR (Type 242), reject when pressure is less than 700 mb + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 242 + test variables: + - name: MetaData/pressure + minvalue: 70000. + action: + name: reject + + # cloud-top WV (Type 250), reject when pressure greater than 399 mb. + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 250 + test variables: + - name: MetaData/pressure + maxvalue: 39900. + action: + name: reject + + # JMA IR (252) reject when pressure between 499 and 801 mb. + # PERFORM ACTION: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/pressure + minvalue: 49901. + maxvalue: 80099. + - variable: ObsType/windEastward + is_in: 252 + action: + name: reject + + # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation + # Turn this check off for now. + # Need to check if troposphere pressure was implemented correctly in fv3-jed + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + reference: GeoVaLs/tropopause_pressure + value: MetaData/pressure + minvalue: -5000. # 50 hPa above tropopause level, negative p-diff + action: + name: reject + + # All satwinds must adjust errors based on ObsErrorFactorPressureCheck + # prior to the SPDB check (i.e. the gross-error check). The gross-error + # check uses the adjusted errors for error-bound tightening and rejection, + # so this check has to come first. This check will inflate errors for obs + # that are too close to either the model top or bottom. + # Notes (eliu): GMAO added a required parameter: adjusted_error_name. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + surface_obs: false + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All satwinds subject to a gross-error check that contains significant + # modifiers for satwinds with a negative speed-bias. ALL wind gross-error + # checks are currently being done by the SatWindsSPDBCheck. + # CLEARED + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, + 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, + 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, + 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, + 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, + 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, + 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, + 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, + 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don"t want to inflate errors for duplication if one + # of the duplicates should be rejected. + # Notes (eliu): ObsErrorFactorDuplicateCheck obsfunction requires PreUseFlag (usage parameter from read_satwnd.f90). + # : Turn off duplicate check for now. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + # We are extending this to an additional filter that inflates final ob-errors across-the-board by + # 1/0.8 = 1.25. This is caused by the GSI value of nvqc being set to .true. in the global operational + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this nvqc functionality were to be switched off (i.e. if variational qc were to be turned off), + # you would want to remove this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + # End of Filters + + # Observation Localizations (LocalEnsembleDA) + # ------------------------------------------- + obs localizations: + - localization method: Horizontal Gaspari-Cohn + lengthscale: 1250e3 + max nobs: 10000 + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/scatwind_ascat_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/scatwind_ascat_metop-b.yaml.j2 new file mode 100644 index 000000000..a35d6dce7 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/scatwind_ascat_metop-b.yaml.j2 @@ -0,0 +1,323 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: ascatw_ascat_metop-b + distribution: + name: Halo + halo size: 1250e3 + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + # Observation Operator + # -------------------- + obs operator: + name: VertInterp + # Use height vertical coordinate first + # vertical coordinate: geometric_height + vertical coordinate: geopotential_height + observation vertical coordinate group: DerivedVariables + observation vertical coordinate: adjustedHeight + interpolation method: linear + hofx scaling field: SurfaceWindScalingHeight + hofx scaling field group: DerivedVariables + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: VertInterp + vertical coordinate: geopotential_height + observation vertical coordinate group: DerivedVariables + observation vertical coordinate: adjustedHeight + interpolation method: linear + + # Observation Pre Filters (QC) + # ---------------------------- + obs pre filters: + - filter: Gaussian Thinning + horizontal_mesh: 75 + use_reduced_horizontal_grid: true + round_horizontal_bin_count_to_nearest: true + partition_longitude_bins_using_mesh: true + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Apply variable changes needed for rescaled height coordinate + - filter: Variable Transforms + Transform: AdjustedHeightCoordinate + SkipWhenNoObs: false + + # Apply variable changes needed for wind scaling + - filter: Variable Transforms + Transform: SurfaceWindScalingHeight + SkipWhenNoObs: false + + # Assign the initial observation error (constant value, 1.5 m/s right now). + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error parameter: 1.5 + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + # Reject all obs with PreQC mark already set above 3 + # NOTE: All scatwinds have an automatic PreQC mark of 2 (hard-wired default from GSI) + # - filter: PreQC + # maxvalue: 3 + # action: + # name: reject + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Reject all ASCAT (Type 290) winds with tsavg <= 273.0 (surface temperature) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 290 + # - variable: GeoVaLs/surface_temperature + - variable: GeoVaLs/surface_temperature_where_land + maxvalue: 273. + action: + name: reject + + # Reject all ASCAT (Type 290) winds with isflg /= 0 (non-water surface) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 290 + - variable: GeoVaLs/water_area_fraction + maxvalue: 0.99 + action: + name: reject + + # Reject ASCAT (Type 290) when observed component deviates from background by more than 5.0 m/s + # NOTE: This check can reject a u- or v-component of the same observation independently, which + # is fundamentally different from how GSI rejects obs (both components are rejected if + # either component fails a check). + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Arithmetic + options: + variables: + - name: ObsValue/windEastward + - name: HofX/windEastward + coefs: [1.0, -1.0] + minvalue: -5.0 + maxvalue: 5.0 + + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Arithmetic + options: + variables: + - name: ObsValue/windNorthward + - name: HofX/windNorthward + coefs: [1.0, -1.0] + minvalue: -5.0 + maxvalue: 5.0 + + # Reject OSCAT (Type 291) when observed component deviates from background by more than 6.0 m/s + # NOTE: This check can reject a u- or v-component of the same observation independently, which + # is fundamentally different from how GSI rejects obs (both components are rejected if + # either component fails a check). + - filter: Background Check + filter variables: + - name: windEastward + - name: windNorthward + threshold: 6. + absolute threshold: 6. + where: + - variable: ObsType/windEastward + is_in: 291 + action: + name: reject + + # Reject ASCAT (Type 290) when ambiguity check fails (returned value is negative) + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 290 + test variables: + - name: ObsFunction/ScatWindsAmbiguityCheck + options: + minimum_uv: 0.0001 # hard-coding a minimum-uv for transparancy, want this to basically be zero + maxvalue: 0. + action: + name: reject + + # All scatwinds must adjust errors based on ObsErrorFactorPressureCheck. + # This check will inflate errors for obs that are too close to either + # the model top or bottom. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 290-291 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 290-291 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All scatwinds subject to a gross error check. This is contained within + # the WindsSPDBCheck, although it is not exclusive to satwinds. + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [290, 291] + cgross: [5.0, 5.0] + error_min: [1.4, 1.4] + error_max: [6.1, 6.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [290, 291] + cgross: [5.0, 5.0] + error_min: [1.4, 1.4] + error_max: [6.1, 6.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don"t want to inflate errors for duplication if one + # of the duplicates should be rejected. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + # There is no across-the-board inflation for nvqc=.true. for scatwinds, presumably because for + # this inflation to take place both nvqc must be .true. AND ibeta must be >0, see: + # https://github.com/NOAA-EMC/GSI/blob/14ae595af1b03471287d322596d35c0665336e95/src/gsi/setupw.f90#L1229 + # GSI settings must have ibeta>0 for satwinds, but not for scatwinds. + # + # If the ibeta settings for scatwinds were to change while nvqc remained .true., we would extend YAML to + # an additional filter that inflates final ob-errors across-the-board by 1/0.8 = 1.25. NOTE: the nvqc setting + # is defaulted to .false. in GSI code, but is overridden in global operational configuration. See: + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this functionality were to be activated for scatwinds, you would want to include this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 290-291 + # action: + # name: inflate error + # inflation factor: 1.25 + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 290-291 + # action: + # name: inflate error + # inflation factor: 1.25 + # END OF FILTERS# + + # Observation Localizations (LocalEnsembleDA) + # ------------------------------------------- + obs localizations: + - localization method: Horizontal Gaspari-Cohn + lengthscale: 1250e3 + max nobs: 10000 + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/sondes.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/sondes.yaml.j2 new file mode 100644 index 000000000..650be3296 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/sondes.yaml.j2 @@ -0,0 +1,59 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: sondes + distribution: + name: Halo + halo size: 1250e3 + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward, airTemperature] + + # Observation Operator + # -------------------- + obs operator: + name: VertInterp + + # Observation Error + # ----------------- + obs error: + covariance model: diagonal + + # Observation Filters (QC) + # ------------------------ + obs filters: + - filter: PreQC + maxvalue: 3 + - filter: Background Check + filter variables: + - name: windEastward + - name: windNorthward + - name: airTemperature + threshold: 2.0 + + # Observation Localizations (LocalEnsembleDA) + # ------------------------------------------- + obs localizations: + - localization method: Horizontal Gaspari-Cohn + lengthscale: 1250e3 + max nobs: 10000 + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/aircraft.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/aircraft.yaml.j2 index 210ca82ad..22b65c043 100644 --- a/parm/jcb-gdas/observations/atmosphere/aircraft.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/aircraft.yaml.j2 @@ -1,10 +1,9 @@ -- obs space: +- + + # Observation Space (I/O) + # ----------------------- + obs space: name: Aircraft - {% if algorithm == "local_ensemble_da" %} - distribution: - name: Halo - halo size: 1250e3 - {% endif %} obsdatain: engine: type: H5File @@ -20,64 +19,62 @@ io pool: max pool size: 1 simulated variables: [windEastward, windNorthward, airTemperature, specificHumidity] - - {% if algorithm == "local_ensemble_da" %} - obs localizations: - - localization method: Horizontal Gaspari-Cohn - lengthscale: 1250e3 - max nobs: 10000] - {% endif %} - # + + # Observation Operator + # -------------------- obs operator: name: VertInterp # + + # Observation Filters (QC) + # ------------------------ obs filters: # Observation Range Sanity Check - - filter: Bounds Check - filter variables: - - name: airTemperature - minvalue: 195 - maxvalue: 327 - action: - name: reject + - filter: Bounds Check + filter variables: + - name: airTemperature + minvalue: 195 + maxvalue: 327 + action: + name: reject # - - filter: Bounds Check - filter variables: - - name: specificHumidity - minvalue: 1.0E-7 - maxvalue: 0.034999999 - action: - name: reject + - filter: Bounds Check + filter variables: + - name: specificHumidity + minvalue: 1.0E-7 + maxvalue: 0.034999999 + action: + name: reject # - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - minvalue: -130 - maxvalue: 130 - action: - name: reject + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130 + maxvalue: 130 + action: + name: reject # - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - test variables: - - name: ObsFunction/Velocity - maxvalue: 130.0 - action: - name: reject + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130.0 + action: + name: reject # Reject when pressure is less than 126 mb. - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - test variables: - - name: MetaData/pressure - minvalue: 12600 - action: - name: reject + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: MetaData/pressure + minvalue: 12600 + action: + name: reject # # Reject all obs with PreQC mark already set above 3 # - filter: PreQC @@ -90,94 +87,98 @@ #-------------------------------------------------------------------------------------------------------------------- # # Begin by assigning all ObsError to a constant value. These will get overwritten (as needed) for specific types. - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - action: - name: assign error - error parameter: 2.0 # 2.0 m/s + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error parameter: 2.0 # 2.0 m/s # Assign intial ObsError specific to AIREP/ACARS - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - action: - name: assign error - error parameter: 3.6 # 3.6 m/s - where: - - variable: - name: ObsType/windEastward - is_in: 230 + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error parameter: 3.6 # 3.6 m/s + where: + - variable: + name: ObsType/windEastward + is_in: 230 # Assign intial ObsError specific to AMDAR - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - action: - name: assign error - error parameter: 3.0 # 3.0 m/s - where: - - variable: - name: ObsType/windEastward - is_in: 231 + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error parameter: 3.0 # 3.0 m/s + where: + - variable: + name: ObsType/windEastward + is_in: 231 # Assign intial ObsError specific to MDCRS - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - action: - name: assign error - error parameter: 2.5 # 2.5 m/s - where: - - variable: - name: ObsType/windEastward - is_in: 233 + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error parameter: 2.5 # 2.5 m/s + where: + - variable: + name: ObsType/windEastward + is_in: 233 # - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [100000, 95000, 80000, 65000, 60000, 55000, 50000, 45000, 40000, 35000, 30000, 25000, 20000, 15000, 10000] #Pressure (Pa) - errors: [1.4, 1.5, 1.6, 1.8, 1.9, 2.0, 2.1, 2.3, 2.6, 2.8, 3.0, 3.2, 2.7, 2.4, 2.1] + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [100000, 95000, 80000, 65000, 60000, 55000, 50000, 45000, 40000, + 35000, 30000, 25000, 20000, 15000, 10000] #Pressure (Pa) + errors: [1.4, 1.5, 1.6, 1.8, 1.9, 2.0, 2.1, 2.3, 2.6, 2.8, 3.0, 3.2, 2.7, + 2.4, 2.1] # Assign the initial ObsError, based on height/pressure for RECON aircraft - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [70000, 65000, 60000, 55000, 50000, 45000, 40000, 35000, 30000, 25000, 20000, 15000, 10000, 7500, 5000] - errors: [2.4, 2.5, 2.6, 2.7, 2.8, 2.95, 3.1, 3.25, 3.4, 3.175, 2.95, 2.725, 2.5, 2.6, 2.7] - where: - - variable: - name: ObsType/windEastward - is_in: 232 - # Reject when difference of wind direction is more than 50 degrees. - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - test variables: - - name: ObsFunction/WindDirAngleDiff + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear options: - minimum_uv: 3.5 - maxvalue: 50.0 - action: - name: reject - defer to post: true + xvar: + name: MetaData/pressure + xvals: [70000, 65000, 60000, 55000, 50000, 45000, 40000, 35000, 30000, 25000, + 20000, 15000, 10000, 7500, 5000] + errors: [2.4, 2.5, 2.6, 2.7, 2.8, 2.95, 3.1, 3.25, 3.4, 3.175, 2.95, 2.725, + 2.5, 2.6, 2.7] + where: + - variable: + name: ObsType/windEastward + is_in: 232 + # Reject when difference of wind direction is more than 50 degrees. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/WindDirAngleDiff + options: + minimum_uv: 3.5 + maxvalue: 50.0 + action: + name: reject + defer to post: true # When multiple obs exist within a single vertical model level, inflate ObsError # - filter: Perform Action # filter variables: @@ -204,77 +205,79 @@ # defer to post: true # # Gross error check with (O - B) / ObsError greater than threshold. - - filter: Background Check - filter variables: - - name: windEastward - - name: windNorthward - threshold: 6.0 - absolute threshold: 19.0 - action: - name: reject + - filter: Background Check + filter variables: + - name: windEastward + - name: windNorthward + threshold: 6.0 + absolute threshold: 19.0 + action: + name: reject # #-------------------------------------------------------------------------------------------------------------------- # Temperature #-------------------------------------------------------------------------------------------------------------------- # # Begin by assigning all ObsError to a constant value. These will get overwritten for specific types. - - filter: Perform Action - filter variables: - - name: airTemperature - action: - name: assign error - error parameter: 2.0 # 2.0 K + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: assign error + error parameter: 2.0 # 2.0 K # Assign the initial observation error, based on pressure (for AIREP/ACARS; itype=130) - - filter: Perform Action - filter variables: - - name: airTemperature - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [100000, 95000, 90000, 85000, 80000] - errors: [2.5, 2.3, 2.1, 1.9, 1.7] - where: - - variable: - name: ObsType/airTemperature - is_in: 130 + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [100000, 95000, 90000, 85000, 80000] + errors: [2.5, 2.3, 2.1, 1.9, 1.7] + where: + - variable: + name: ObsType/airTemperature + is_in: 130 # Assign the initial observation error, based on pressure (for AMDAR and MDCRS; itype=131,133) - - filter: Perform Action - filter variables: - - name: airTemperature - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [100000, 95000, 90000, 85000, 80000] - errors: [1.4706, 1.3529, 1.2353, 1.1176, 1.0] - where: - - variable: - name: ObsType/airTemperature - is_in: 131,133 + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [100000, 95000, 90000, 85000, 80000] + errors: [1.4706, 1.3529, 1.2353, 1.1176, 1.0] + where: + - variable: + name: ObsType/airTemperature + is_in: 131,133 # Assign the initial observation error, based on pressure (for RECON aircraft; itype=132) - - filter: Perform Action - filter variables: - - name: airTemperature - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [100000, 95000, 90000, 85000, 35000, 30000, 25000, 20000, 15000, 10000, 7500, 5000, 4000, 3200, 2000, 1000] - errors: [1.2, 1.1, 0.9, 0.8, 0.8, 0.9, 1.2, 1.2, 1.0, 0.8, 0.8, 0.9, 0.95, 1.0, 1.25, 1.5] - where: - - variable: - name: ObsType/airTemperature - is_in: 132 + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [100000, 95000, 90000, 85000, 35000, 30000, 25000, 20000, 15000, + 10000, 7500, 5000, 4000, 3200, 2000, 1000] + errors: [1.2, 1.1, 0.9, 0.8, 0.8, 0.9, 1.2, 1.2, 1.0, 0.8, 0.8, 0.9, 0.95, + 1.0, 1.25, 1.5] + where: + - variable: + name: ObsType/airTemperature + is_in: 132 # When multiple obs exist within a single vertical model level, inflate ObsError # - filter: Perform Action # filter variables: @@ -287,40 +290,41 @@ # test QCflag: PreQC # inflate variables: [airTemperature] # defer to post: true - # Gross error check with (O - B) / ObsError greater than threshold. - - filter: Background Check - filter variables: - - name: airTemperature - threshold: 7.0 - absolute threshold: 8.0 - action: - name: reject + - filter: Background Check + filter variables: + - name: airTemperature + threshold: 7.0 + absolute threshold: 8.0 + action: + name: reject # #-------------------------------------------------------------------------------------------------------------------- # Moisture #-------------------------------------------------------------------------------------------------------------------- # # Assign the initial observation error, based on height/pressure ONLY MDCRS - - filter: Perform Action - filter variables: - - name: specificHumidity - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [110000, 105000, 100000, 95000, 90000, 85000, 80000, 75000, 70000, 65000, 60000, 55000, - 50000, 45000, 40000, 35000, 30000, 25000, 20000, 15000, 10000, 7500, 5000, 4000, 3000] - errors: [.19455, .19062, .18488, .17877, .17342, .16976, .16777, .16696, .16605, .16522, .16637, .17086, - .17791, .18492, .18996, .19294, .19447, .19597, .19748, .19866, .19941, .19979, .19994, .19999, .2] - scale_factor_var: ObsValue/specificHumidity - where: - - variable: - name: ObsType/specificHumidity - is_in: 133 + - filter: Perform Action + filter variables: + - name: specificHumidity + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000, 105000, 100000, 95000, 90000, 85000, 80000, 75000, 70000, + 65000, 60000, 55000, 50000, 45000, 40000, 35000, 30000, 25000, 20000, + 15000, 10000, 7500, 5000, 4000, 3000] + errors: [.19455, .19062, .18488, .17877, .17342, .16976, .16777, .16696, + .16605, .16522, .16637, .17086, .17791, .18492, .18996, .19294, .19447, + .19597, .19748, .19866, .19941, .19979, .19994, .19999, .2] + scale_factor_var: ObsValue/specificHumidity + where: + - variable: + name: ObsType/specificHumidity + is_in: 133 # When multiple obs exist within a single vertical model level, inflate ObsError # - filter: Perform Action # filter variables: @@ -333,19 +337,20 @@ # test QCflag: PreQC # inflate variables: [specificHumidity] # defer to post: true - # Gross error check with (O - B) / ObsError greater than threshold. - - filter: Background Check - filter variables: - - name: specificHumidity - threshold: 8.0 - action: - name: reject + - filter: Background Check + filter variables: + - name: specificHumidity + threshold: 8.0 + action: + name: reject -{% if algorithm == "test_obs_filters" %} + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- geovals: filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" - passedBenchmark: 107747 -{% endif %} + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 index 9ee108537..63119f938 100644 --- a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 @@ -1,5 +1,9 @@ -- obs space: - name: AMSU-A NOAA-19 +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: AMSUA N19 obsdatain: engine: type: H5File @@ -11,16 +15,22 @@ io pool: max pool size: 1 simulated variables: [brightnessTemperature] - channels: &simulated_channels {{ get_satellite_simulated_channels(observation_from_jcb) }} + channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} + + # Observation Operator + # -------------------- obs operator: name: CRTM - Absorbers: [H2O,O3] + Absorbers: [H2O, O3] Clouds: [Water, Ice] Cloud_Fraction: 1.0 obs options: - Sensor_ID: {{observation_from_jcb}} + Sensor_ID: &{{observation_from_jcb}}_sensor_id amsua_n19 EndianType: little_endian - CoefficientPath: {{crtm_coefficient_path}} + CoefficientPath: "{{crtm_coefficient_path}}" + + # Observation Bias Correction (VarBC) + # ----------------------------------- obs bias: input file: "{{atm_obsbiasin_path}}/{{atm_obsbiasin_prefix}}{{observation_from_jcb}}{{atm_obsbiasin_suffix}}" output file: "{{atm_obsbiasout_path}}/{{atm_obsbiasout_prefix}}{{observation_from_jcb}}{{atm_obsbiasout_suffix}}" @@ -51,28 +61,35 @@ ratio: 1.1 ratio for small dataset: 2.0 output file: "{{atm_obsbiasout_path}}/{{atm_obsbiascovout_prefix}}{{observation_from_jcb}}{{atm_obsbiascovout_suffix}}" + + # Observation Filters (QC) + # ------------------------ obs filters: - filter: BlackList filter variables: - name: brightnessTemperature - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels action: name: assign error error function: name: ObsFunction/ObsErrorModelRamp - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels options: - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels xvar: name: ObsFunction/CLWRetSymmetricMW options: clwret_ch238: 1 clwret_ch314: 2 clwret_types: [ObsValue, HofX] - x0: &x0 {{ get_satellite_channel_dep_variable(observation_from_jcb, "x0") }} - x1: &x1 {{ get_satellite_channel_dep_variable(observation_from_jcb, "x1") }} - err0: &err0 {{ get_satellite_channel_dep_variable(observation_from_jcb, "err0") }} - err1: &err1 {{ get_satellite_channel_dep_variable(observation_from_jcb, "err1") }} + x0: [0.050, 0.030, 0.030, 0.020, 0.000, 0.100, 0.000, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.030] + x1: [0.600, 0.450, 0.400, 0.450, 1.000, 1.500, 0.000, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.200] + err0: [2.500, 2.200, 2.000, 0.550, 0.300, 0.230, 0.230, 0.250, 0.250, 0.350, + 0.400, 0.550, 0.800, 3.000, 3.500] + err1: [20.000, 18.000, 12.000, 3.000, 0.500, 0.300, 0.230, 0.250, 0.250, + 0.350, 0.400, 0.550, 0.800, 3.000, 18.000] # CLW Retrieval Check - filter: Bounds Check filter variables: @@ -105,13 +122,14 @@ - filter: Bounds Check filter variables: - name: brightnessTemperature - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels test variables: - name: ObsFunction/HydrometeorCheckAMSUA - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels options: - channels: *simulated_channels - obserr_clearsky: &obserr_clearsky {{ get_satellite_channel_dep_variable(observation_from_jcb, "obserr_clearsky") }} + channels: *{{observation_from_jcb}}_simulated_channels + obserr_clearsky: [2.500, 2.200, 2.000, 0.550, 0.300, 0.230, 0.230, 0.250, + 0.250, 0.350, 0.400, 0.550, 0.800, 3.000, 3.500] clwret_function: name: ObsFunction/CLWRetMW options: @@ -120,19 +138,23 @@ clwret_types: [ObsValue] obserr_function: name: ObsFunction/ObsErrorModelRamp - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels options: - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels xvar: name: ObsFunction/CLWRetSymmetricMW options: clwret_ch238: 1 clwret_ch314: 2 clwret_types: [ObsValue, HofX] - x0: *x0 - x1: *x1 - err0: *err0 - err1: *err1 + x0: [0.050, 0.030, 0.030, 0.020, 0.000, 0.100, 0.000, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.030] + x1: [0.600, 0.450, 0.400, 0.450, 1.000, 1.500, 0.000, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.200] + err0: [2.500, 2.200, 2.000, 0.550, 0.300, 0.230, 0.230, 0.250, 0.250, + 0.350, 0.400, 0.550, 0.800, 3.000, 3.500] + err1: [20.000, 18.000, 12.000, 3.000, 0.500, 0.300, 0.230, 0.250, 0.250, + 0.350, 0.400, 0.550, 0.800, 3.000, 18.000] maxvalue: 0.0 action: name: reject @@ -140,56 +162,56 @@ - filter: BlackList filter variables: - name: brightnessTemperature - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels action: name: inflate error inflation variable: name: ObsFunction/ObsErrorFactorTopoRad - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels options: - sensor: {{observation_from_jcb}} - channels: *simulated_channels + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels # Transmittnace Top Check - filter: BlackList filter variables: - name: brightnessTemperature - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels action: name: inflate error inflation variable: name: ObsFunction/ObsErrorFactorTransmitTopRad - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels options: - sensor: {{observation_from_jcb}} - channels: *simulated_channels + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels # Surface Jacobian check - filter: BlackList filter variables: - name: brightnessTemperature - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels action: name: inflate error inflation variable: name: ObsFunction/ObsErrorFactorSurfJacobianRad - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels options: - sensor: {{observation_from_jcb}} - channels: *simulated_channels + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels obserr_demisf: [0.010, 0.020, 0.015, 0.020, 0.200] obserr_dtempf: [0.500, 2.000, 1.000, 2.000, 4.500] # Situation dependent Check - filter: BlackList filter variables: - name: brightnessTemperature - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels action: name: inflate error inflation variable: name: ObsFunction/ObsErrorFactorSituDependMW - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels options: - sensor: {{observation_from_jcb}} - channels: *simulated_channels + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels clwobs_function: name: ObsFunction/CLWRetMW options: @@ -213,9 +235,9 @@ bias_application: HofX clwmatchidx_function: name: ObsFunction/CLWMatchIndexMW - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels options: - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels clwobs_function: name: ObsFunction/CLWRetMW options: @@ -229,54 +251,60 @@ clwret_ch314: 2 clwret_types: [HofX] bias_application: HofX - clwret_clearsky: {{ get_satellite_channel_dep_variable(observation_from_jcb, "clwret_clearsky") }} + clwret_clearsky: [0.050, 0.030, 0.030, 0.020, 0.000, 0.100, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.030] obserr_function: name: ObsFunction/ObsErrorModelRamp - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels options: - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels xvar: name: ObsFunction/CLWRetSymmetricMW options: clwret_ch238: 1 clwret_ch314: 2 clwret_types: [ObsValue, HofX] - x0: *x0 - x1: *x1 - err0: *err0 - err1: *err1 - obserr_clearsky: *obserr_clearsky + x0: [0.050, 0.030, 0.030, 0.020, 0.000, 0.100, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.000, 0.030] + x1: [0.600, 0.450, 0.400, 0.450, 1.000, 1.500, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.000, 0.200] + err0: [2.500, 2.200, 2.000, 0.550, 0.300, 0.230, 0.230, 0.250, 0.250, + 0.350, 0.400, 0.550, 0.800, 3.000, 3.500] + err1: [20.000, 18.000, 12.000, 3.000, 0.500, 0.300, 0.230, 0.250, 0.250, + 0.350, 0.400, 0.550, 0.800, 3.000, 18.000] + obserr_clearsky: [2.500, 2.200, 2.000, 0.550, 0.300, 0.230, 0.230, 0.250, + 0.250, 0.350, 0.400, 0.550, 0.800, 3.000, 3.500] # Gross check - filter: Background Check filter variables: - name: brightnessTemperature - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels function absolute threshold: - name: ObsFunction/ObsErrorBoundMW - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels options: - sensor: {{observation_from_jcb}} - channels: *simulated_channels + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels obserr_bound_latitude: name: ObsFunction/ObsErrorFactorLatRad options: latitude_parameters: [25.0, 0.25, 0.04, 3.0] obserr_bound_transmittop: name: ObsFunction/ObsErrorFactorTransmitTopRad - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels options: - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels obserr_bound_topo: name: ObsFunction/ObsErrorFactorTopoRad - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels options: - channels: *simulated_channels - sensor: {{observation_from_jcb}} + channels: *{{observation_from_jcb}}_simulated_channels + sensor: *{{observation_from_jcb}}_sensor_id obserr_function: name: ObsFunction/ObsErrorModelRamp - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels options: - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels xvar: name: ObsFunction/CLWRetSymmetricMW options: @@ -284,25 +312,30 @@ clwret_ch314: 2 clwret_types: [ObsValue, HofX] bias_application: HofX - x0: *x0 - x1: *x1 - err0: *err0 - err1: *err1 - obserr_bound_max: {{ get_satellite_channel_dep_variable(observation_from_jcb, "obserr_bound_max") }} + x0: [0.050, 0.030, 0.030, 0.020, 0.000, 0.100, 0.000, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.030] + x1: [0.600, 0.450, 0.400, 0.450, 1.000, 1.500, 0.000, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.200] + err0: [2.500, 2.200, 2.000, 0.550, 0.300, 0.230, 0.230, 0.250, 0.250, + 0.350, 0.400, 0.550, 0.800, 3.000, 3.500] + err1: [20.000, 18.000, 12.000, 3.000, 0.500, 0.300, 0.230, 0.250, 0.250, + 0.350, 0.400, 0.550, 0.800, 3.000, 18.000] + obserr_bound_max: [4.5, 4.5, 4.5, 2.5, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.5, + 3.5, 4.5, 4.5, 4.5] action: name: reject # Inter-channel check - filter: Bounds Check filter variables: - name: brightnessTemperature - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels test variables: - name: ObsFunction/InterChannelConsistencyCheck - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels options: - channels: *simulated_channels - sensor: {{observation_from_jcb}} - use_flag: &active_channels {{ get_satellite_active_channels(observation_from_jcb) }} + channels: *{{observation_from_jcb}}_simulated_channels + sensor: *{{observation_from_jcb}}_sensor_id + use_flag: &{{observation_from_jcb}}_active_channels {{ get_satellite_variable(observation_from_jcb, "active") }} maxvalue: 1.0e-12 action: name: reject @@ -310,22 +343,24 @@ - filter: Bounds Check filter variables: - name: brightnessTemperature - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels test variables: - name: ObsFunction/ChannelUseflagCheckRad - channels: *simulated_channels + channels: *{{observation_from_jcb}}_simulated_channels options: - sensor: {{observation_from_jcb}} - channels: *simulated_channels - use_flag: *active_channels + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: *{{observation_from_jcb}}_active_channels minvalue: 1.0e-12 action: name: reject -{% if algorithm == "test_obs_filters" %} + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- geovals: filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" - passedBenchmark: 107747 -{% endif %} + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 new file mode 100644 index 000000000..fe492cf20 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 @@ -0,0 +1,537 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: ATMS N20 + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [brightnessTemperature] + channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} + + # Observation Operator + # -------------------- + obs operator: + name: CRTM + Absorbers: [H2O, O3, CO2] + Clouds: [Water, Ice] + Cloud_Fraction: 1.0 + Cloud_Seeding: true + obs options: + Sensor_ID: &{{observation_from_jcb}}_sensor_id atms_n20 + EndianType: little_endian + CoefficientPath: "{{crtm_coefficient_path}}" + linear obs operator: + Absorbers: [H2O, O3] + + # Observation Bias Correction (VarBC) + # ----------------------------------- + obs bias: + input file: "{{atm_obsbiasin_path}}/{{atm_obsbiasin_prefix}}{{observation_from_jcb}}{{atm_obsbiasin_suffix}}" + output file: "{{atm_obsbiasout_path}}/{{atm_obsbiasout_prefix}}{{observation_from_jcb}}{{atm_obsbiasout_suffix}}" + variational bc: + predictors: + - name: constant + - name: lapse_rate + order: 2 + tlapse: &{{observation_from_jcb}}_tlapse "{{atm_obsbiasin_path}}/{{atm_obstlapsein_prefix}}{{observation_from_jcb}}{{atm_obstlapsein_suffix}}" + - name: lapse_rate + tlapse: *{{observation_from_jcb}}_tlapse + - name: emissivity + - name: scan_angle + order: 4 + - name: scan_angle + order: 3 + - name: scan_angle + order: 2 + - name: scan_angle + covariance: + minimal required obs number: 20 + variance range: [1.0e-6, 10.0] + step size: 1.0e-4 + largest analysis variance: 10000.0 + prior: + input file: "{{atm_obsbiasin_path}}/{{atm_obsbiascovin_prefix}}{{observation_from_jcb}}{{atm_obsbiascovin_suffix}}" + inflation: + ratio: 1.1 + ratio for small dataset: 2.0 + output file: "{{atm_obsbiasout_path}}/{{atm_obsbiascovout_prefix}}{{observation_from_jcb}}{{atm_obsbiascovout_suffix}}" + + # Observation Pre Filters (QC) + # ---------------------------- + obs pre filters: + # Step 0-A: Create Diagnostic Flags + - filter: Create Diagnostic Flags + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + flags: + - name: ScanEdgeRemoval + initial value: false + force reinitialization: false + - name: Thinning + initial value: false + force reinitialization: false + - name: CLWRetrievalCheck + initial value: false + force reinitialization: false + - name: WindowChannelExtremeResidual + initial value: false + force reinitialization: false + - name: HydrometeorCheck + initial value: false + force reinitialization: false + - name: GrossCheck + initial value: false + force reinitialization: false + - name: InterChannelConsistency + initial value: false + force reinitialization: false + - name: UseflagCheck + initial value: false + force reinitialization: false + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Step 0-B: Calculate derived variables + # Calculate CLW retrieved from observation + - filter: Variable Assignment + assignments: + - name: CLWRetFromObs@DerivedMetaData + type: float + function: + name: CLWRetMW@ObsFunction + options: + clwret_ch238: 1 + clwret_ch314: 2 + clwret_types: [ObsValue] + # Calculate CLW retrieved from observation + - filter: Variable Assignment + assignments: + - name: CLWRetFromBkg@DerivedMetaData + type: float + function: + name: CLWRetMW@ObsFunction + options: + clwret_ch238: 1 + clwret_ch314: 2 + clwret_types: [HofX] + # Calculate symmetric retrieved CLW + - filter: Variable Assignment + assignments: + - name: CLWRetSymmetric@DerivedMetaData + type: float + value: 1000.0 + + - filter: Variable Assignment + where: + - variable: + name: CLWRetFromObs@DerivedMetaData + minvalue: 0. + maxvalue: 999. + - variable: + name: CLWRetFromBkg@DerivedMetaData + minvalue: 0. + maxvalue: 999. + where operator: and + assignments: + - name: CLWRetSymmetric@DerivedMetaData + type: float + function: + name: Arithmetic@ObsFunction + options: + variables: + - name: CLWRetFromObs@DerivedMetaData + - name: CLWRetFromBkg@DerivedMetaData + total coefficient: 0.5 + + # Calculate scattering index from observation + - filter: Variable Assignment + assignments: + - name: SIRetFromObs@DerivedMetaData + type: float + function: + name: SCATRetMW@ObsFunction + options: + scatret_ch238: 1 + scatret_ch314: 2 + scatret_ch890: 16 + scatret_types: [ObsValue] + # Calculate CLW obs/bkg match index + - filter: Variable Assignment + assignments: + - name: CLWMatchIndex@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: CLWMatchIndexMW@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + clwobs_function: + name: CLWRetFromObs@DerivedMetaData + clwbkg_function: + name: CLWRetFromBkg@DerivedMetaData + clwret_clearsky: [0.030, 0.030, 0.030, 0.020, 0.030, 0.080, 0.150, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.020, 0.030, 0.030, + 0.030, 0.030, 0.050, 0.100] + # Calculate symmetric observation error + - filter: Variable Assignment + assignments: + - name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorModelRamp@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + xvar: + name: CLWRetSymmetric@DerivedMetaData + x0: [0.030, 0.030, 0.030, 0.020, 0.030, 0.080, 0.150, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.000, 0.020, 0.030, 0.030, 0.030, 0.030, + 0.050, 0.100] + x1: [0.350, 0.380, 0.400, 0.450, 0.500, 1.000, 1.000, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.000, 0.350, 0.500, 0.500, 0.500, 0.500, + 0.500, 0.500] + err0: [4.500, 4.500, 4.500, 2.500, 0.550, 0.300, 0.300, 0.400, 0.400, 0.400, + 0.450, 0.450, 0.550, 0.800, 4.000, 4.000, 4.000, 3.500, 3.000, 3.000, + 3.000, 3.000] + err1: [20.000, 25.000, 12.000, 7.000, 3.500, 3.000, 0.800, 0.400, 0.400, + 0.400, 0.450, 0.450, 0.550, 0.800, 4.000, 19.000, 30.000, 25.000, 16.500, + 12.000, 9.000, 6.500] + # Calculate Innovation@DerivedMetaData + - filter: Variable Assignment + assignments: + - name: Innovation@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsFunction/Arithmetic + channels: *{{observation_from_jcb}}_simulated_channels + options: + variables: + - name: brightnessTemperature@ObsValue + channels: *{{observation_from_jcb}}_simulated_channels + - name: brightnessTemperature@HofX + channels: *{{observation_from_jcb}}_simulated_channels + coefs: [1, -1] + # Step 0-C: Assign Initial All-Sky Observation Error + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: assign error + error function: + name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + # Step 1: Remove Observations from the Edge of the Scan + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: 1-22 + where: + - variable: + name: MetaData/sensorScanPosition + is_in: 7-90 + actions: + - name: set + flag: ScanEdgeRemoval + - name: reject + + # Step 2: Data Thinning + - filter: Gaussian Thinning + horizontal_mesh: 145 + use_reduced_horizontal_grid: true + distance_norm: geodesic + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + actions: + - name: set + flag: Thinning + - name: reject + + # Step 3A: CLW Retrieval Check (observation_based) + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: 1-7, 16-22 + test variables: + - name: CLWRetFromObs@DerivedMetaData + maxvalue: 999.0 + actions: + - name: set + flag: CLWRetrievalCheck + - name: reject + + # Step 3B: CLW Retrieval Check (background_based) + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: 1-7, 16-22 + test variables: + - name: CLWRetFromBkg@DerivedMetaData + maxvalue: 999.0 + actions: + - name: set + flag: CLWRetrievalCheck + - name: reject + + # Step 4: Window Channel Sanity Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: 1-7, 16, 17-22 + test variables: + - name: Innovation@DerivedMetaData + channels: 1, 2, 5-7, 16 + maxvalue: 200.0 + minvalue: -200.0 + flag all filter variables if any test variable is out of bounds: true + actions: + - name: set + flag: WindowChannelExtremeResidual + - name: reject + + # Step 5: Hydrometeor Check (cloud/precipitation affected chanels) + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/HydrometeorCheckATMS + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: HydrometeorCheckATMS@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_clearsky: [4.500, 4.500, 4.500, 2.500, 0.550, 0.300, 0.300, 0.400, + 0.400, 0.400, 0.450, 0.450, 0.550, 0.800, 4.000, 4.000, 4.000, 3.500, + 3.000, 3.000, 3.000, 3.000] + clwret_function: + name: CLWRetFromObs@DerivedMetaData + obserr_function: + name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: DerivedMetaData/HydrometeorCheckATMS + channels: *{{observation_from_jcb}}_simulated_channels + maxvalue: 0.0 + actions: + - name: set + flag: HydrometeorCheck + ignore: rejected observations + - name: reject + + # Step 6: Observation Error Inflation based on Topography Check + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorTopo@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorFactorTopoRad@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsErrorFactorTopo@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + # Step 7: Obs Error Inflation based on TOA Transmittancec Check + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorTransmitTop@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorFactorTransmitTopRad@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsErrorFactorTransmitTop@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + # Step 8: Observation Error Inflation based on Surface Jacobian Check + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorSurfJacobian@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorFactorSurfJacobianRad@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + obserr_demisf: [0.010, 0.020, 0.015, 0.020, 0.200] + obserr_dtempf: [0.500, 2.000, 1.000, 2.000, 4.500] + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsErrorFactorSurfJacobian@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + # Step 9: Situation Dependent Check + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorSituDepend@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorFactorSituDependMW@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + clwbkg_function: + name: CLWRetFromBkg@DerivedMetaData + clwobs_function: + name: CLWRetFromObs@DerivedMetaData + scatobs_function: + name: SIRetFromObs@DerivedMetaData + clwmatchidx_function: + name: CLWMatchIndex@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + obserr_function: + name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + obserr_clearsky: [4.500, 4.500, 4.500, 2.500, 0.550, 0.300, 0.300, 0.400, + 0.400, 0.400, 0.450, 0.450, 0.550, 0.800, 4.000, 4.000, 4.000, 3.500, + 3.000, 3.000, 3.000, 3.000] + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsErrorFactorSituDepend@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + # Step 10: Gross check + # Remove data if abs(Obs-HofX) > absolute threhold + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorLat@DerivedMetaData + type: float + function: + name: ObsErrorFactorLatRad@ObsFunction + options: + latitude_parameters: [25.0, 0.25, 0.04, 3.0] + - filter: Variable Assignment + assignments: + - name: ObsErrorBound@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorBoundMW@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_latitude: + name: ObsErrorFactorLat@DerivedMetaData + obserr_bound_transmittop: + name: ObsErrorFactorTransmitTop@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_topo: + name: ObsErrorFactorTopo@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + obserr_function: + name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + threhold: 3 + obserr_bound_max: [4.5, 4.5, 3.0, 3.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, + 1.0, 1.0, 2.0, 4.5, 4.5, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0] + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + function absolute threshold: + - name: ObsErrorBound@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + actions: + - name: set + flag: GrossCheck + ignore: rejected observations + - name: reject + + # Step 11: Inter-Channel Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: InterChannelConsistencyCheck@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use passive_bc: true + sensor: *{{observation_from_jcb}}_sensor_id + use_flag: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 4, 1, 1, 1, 1, 1, 1, + 1] + maxvalue: 1.0e-12 + actions: + - name: set + flag: InterChannelConsistency + ignore: rejected observations + - name: reject + + # Step 12: Useflag Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/ChannelUseflagCheckRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 4, 1, 1, 1, 1, 1, 1, + 1] + minvalue: 1.0e-12 + actions: + - name: set + flag: UseflagCheck + ignore: rejected observations + - name: reject + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 new file mode 100644 index 000000000..196bbf471 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 @@ -0,0 +1,555 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: ATMS NPP + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [brightnessTemperature] + channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} + + # Observation Operator + # -------------------- + obs operator: + name: CRTM + Absorbers: [H2O, O3, CO2] + Clouds: [Water, Ice] + Cloud_Fraction: 1.0 + Cloud_Seeding: true + obs options: + Sensor_ID: &{{observation_from_jcb}}_sensor_id atms_npp + EndianType: little_endian + CoefficientPath: "{{crtm_coefficient_path}}" + linear obs operator: + Absorbers: [H2O, O3] + + # Observation Bias Correction (VarBC) + # ----------------------------------- + obs bias: + input file: "{{atm_obsbiasin_path}}/{{atm_obsbiasin_prefix}}{{observation_from_jcb}}{{atm_obsbiasin_suffix}}" + output file: "{{atm_obsbiasout_path}}/{{atm_obsbiasout_prefix}}{{observation_from_jcb}}{{atm_obsbiasout_suffix}}" + variational bc: + predictors: + - name: constant + - name: lapse_rate + order: 2 + tlapse: &{{observation_from_jcb}}_tlapse "{{atm_obsbiasin_path}}/{{atm_obstlapsein_prefix}}{{observation_from_jcb}}{{atm_obstlapsein_suffix}}" + - name: lapse_rate + tlapse: *{{observation_from_jcb}}_tlapse + - name: emissivity + - name: scan_angle + order: 4 + - name: scan_angle + order: 3 + - name: scan_angle + order: 2 + - name: scan_angle + covariance: + minimal required obs number: 20 + variance range: [1.0e-6, 10.0] + step size: 1.0e-4 + largest analysis variance: 10000.0 + prior: + input file: "{{atm_obsbiasin_path}}/{{atm_obsbiascovin_prefix}}{{observation_from_jcb}}{{atm_obsbiascovin_suffix}}" + inflation: + ratio: 1.1 + ratio for small dataset: 2.0 + output file: "{{atm_obsbiasout_path}}/{{atm_obsbiascovout_prefix}}{{observation_from_jcb}}{{atm_obsbiascovout_suffix}}" + + # Observation Pre Filters (QC) + # ---------------------------- + obs pre filters: + # Step 0-A: Create Diagnostic Flags + - filter: Create Diagnostic Flags + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + flags: + - name: ScanEdgeRemoval + initial value: false + force reinitialization: false + - name: Thinning + initial value: false + force reinitialization: false + - name: CLWRetrievalCheck + initial value: false + force reinitialization: false + - name: WindowChannelExtremeResidual + initial value: false + force reinitialization: false + - name: HydrometeorCheck + initial value: false + force reinitialization: false + - name: GrossCheck + initial value: false + force reinitialization: false + - name: InterChannelConsistency + initial value: false + force reinitialization: false + - name: UseflagCheck + initial value: false + force reinitialization: false + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Step 0-B: Calculate derived variables + # Calculate CLW retrieved from observation + - filter: Variable Assignment + assignments: + - name: CLWRetFromObs@DerivedMetaData + type: float + function: + name: CLWRetMW@ObsFunction + options: + clwret_ch238: 1 + clwret_ch314: 2 + clwret_types: [ObsValue] + + # Calculate CLW retrieved from observation + - filter: Variable Assignment + assignments: + - name: CLWRetFromBkg@DerivedMetaData + type: float + function: + name: CLWRetMW@ObsFunction + options: + clwret_ch238: 1 + clwret_ch314: 2 + clwret_types: [HofX] + + # Calculate symmetric retrieved CLW + - filter: Variable Assignment + assignments: + - name: CLWRetSymmetric@DerivedMetaData + type: float + value: 1000.0 + + - filter: Variable Assignment + where: + - variable: + name: CLWRetFromObs@DerivedMetaData + minvalue: 0. + maxvalue: 999. + - variable: + name: CLWRetFromBkg@DerivedMetaData + minvalue: 0. + maxvalue: 999. + where operator: and + assignments: + - name: CLWRetSymmetric@DerivedMetaData + type: float + function: + name: Arithmetic@ObsFunction + options: + variables: + - name: CLWRetFromObs@DerivedMetaData + - name: CLWRetFromBkg@DerivedMetaData + total coefficient: 0.5 + + # Calculate scattering index from observation + - filter: Variable Assignment + assignments: + - name: SIRetFromObs@DerivedMetaData + type: float + function: + name: SCATRetMW@ObsFunction + options: + scatret_ch238: 1 + scatret_ch314: 2 + scatret_ch890: 16 + scatret_types: [ObsValue] + + # Calculate CLW obs/bkg match index + - filter: Variable Assignment + assignments: + - name: CLWMatchIndex@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: CLWMatchIndexMW@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + clwobs_function: + name: CLWRetFromObs@DerivedMetaData + clwbkg_function: + name: CLWRetFromBkg@DerivedMetaData + clwret_clearsky: [0.030, 0.030, 0.030, 0.020, 0.030, 0.080, 0.150, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.020, 0.030, 0.030, + 0.030, 0.030, 0.050, 0.100] + + # Calculate symmetric observation error + - filter: Variable Assignment + assignments: + - name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorModelRamp@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + xvar: + name: CLWRetSymmetric@DerivedMetaData + x0: [0.030, 0.030, 0.030, 0.020, 0.030, 0.080, 0.150, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.000, 0.020, 0.030, 0.030, 0.030, 0.030, + 0.050, 0.100] + x1: [0.350, 0.380, 0.400, 0.450, 0.500, 1.000, 1.000, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.000, 0.350, 0.500, 0.500, 0.500, 0.500, + 0.500, 0.500] + err0: [4.500, 4.500, 4.500, 2.500, 0.550, 0.300, 0.300, 0.400, 0.400, 0.400, + 0.450, 0.450, 0.550, 0.800, 4.000, 4.000, 4.000, 3.500, 3.000, 3.000, + 3.000, 3.000] + err1: [20.000, 25.000, 12.000, 7.000, 3.500, 3.000, 0.800, 0.400, 0.400, + 0.400, 0.450, 0.450, 0.550, 0.800, 4.000, 19.000, 30.000, 25.000, 16.500, + 12.000, 9.000, 6.500] + + # Calculate Innovation@DerivedMetaData + - filter: Variable Assignment + assignments: + - name: Innovation@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsFunction/Arithmetic + channels: *{{observation_from_jcb}}_simulated_channels + options: + variables: + - name: brightnessTemperature@ObsValue + channels: *{{observation_from_jcb}}_simulated_channels + - name: brightnessTemperature@HofX + channels: *{{observation_from_jcb}}_simulated_channels + coefs: [1, -1] + + # Step 0-C: Assign Initial All-Sky Observation Error + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: assign error + error function: + name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 1: Remove Observations from the Edge of the Scan + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: 1-22 + where: + - variable: + name: MetaData/sensorScanPosition + is_in: 7-90 + actions: + - name: set + flag: ScanEdgeRemoval + - name: reject + + # Step 2: Data Thinning + - filter: Gaussian Thinning + horizontal_mesh: 145 + use_reduced_horizontal_grid: true + distance_norm: geodesic + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + actions: + - name: set + flag: Thinning + - name: reject + + # Step 3A: CLW Retrieval Check (observation_based) + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: 1-7, 16-22 + test variables: + - name: CLWRetFromObs@DerivedMetaData + maxvalue: 999.0 + actions: + - name: set + flag: CLWRetrievalCheck + - name: reject + + # Step 3B: CLW Retrieval Check (background_based) + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: 1-7, 16-22 + test variables: + - name: CLWRetFromBkg@DerivedMetaData + maxvalue: 999.0 + actions: + - name: set + flag: CLWRetrievalCheck + - name: reject + + # Step 4: Window Channel Sanity Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: 1-7, 16, 17-22 + test variables: + - name: Innovation@DerivedMetaData + channels: 1, 2, 5-7, 16 + maxvalue: 200.0 + minvalue: -200.0 + flag all filter variables if any test variable is out of bounds: true + actions: + - name: set + flag: WindowChannelExtremeResidual + - name: reject + + # Step 5: Hydrometeor Check (cloud/precipitation affected chanels) + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/HydrometeorCheckATMS + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: HydrometeorCheckATMS@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_clearsky: [4.500, 4.500, 4.500, 2.500, 0.550, 0.300, 0.300, 0.400, + 0.400, 0.400, 0.450, 0.450, 0.550, 0.800, 4.000, 4.000, 4.000, 3.500, + 3.000, 3.000, 3.000, 3.000] + clwret_function: + name: CLWRetFromObs@DerivedMetaData + obserr_function: + name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: DerivedMetaData/HydrometeorCheckATMS + channels: *{{observation_from_jcb}}_simulated_channels + maxvalue: 0.0 + actions: + - name: set + flag: HydrometeorCheck + ignore: rejected observations + - name: reject + + # Step 6: Observation Error Inflation based on Topography Check + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorTopo@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorFactorTopoRad@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsErrorFactorTopo@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 7: Obs Error Inflation based on TOA Transmittancec Check + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorTransmitTop@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorFactorTransmitTopRad@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsErrorFactorTransmitTop@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 8: Observation Error Inflation based on Surface Jacobian Check + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorSurfJacobian@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorFactorSurfJacobianRad@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + obserr_demisf: [0.010, 0.020, 0.015, 0.020, 0.200] + obserr_dtempf: [0.500, 2.000, 1.000, 2.000, 4.500] + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsErrorFactorSurfJacobian@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 9: Situation Dependent Check + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorSituDepend@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorFactorSituDependMW@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + clwbkg_function: + name: CLWRetFromBkg@DerivedMetaData + clwobs_function: + name: CLWRetFromObs@DerivedMetaData + scatobs_function: + name: SIRetFromObs@DerivedMetaData + clwmatchidx_function: + name: CLWMatchIndex@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + obserr_function: + name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + obserr_clearsky: [4.500, 4.500, 4.500, 2.500, 0.550, 0.300, 0.300, 0.400, + 0.400, 0.400, 0.450, 0.450, 0.550, 0.800, 4.000, 4.000, 4.000, 3.500, + 3.000, 3.000, 3.000, 3.000] + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsErrorFactorSituDepend@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 10: Gross check + # Remove data if abs(Obs-HofX) > absolute threhold + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorLat@DerivedMetaData + type: float + function: + name: ObsErrorFactorLatRad@ObsFunction + options: + latitude_parameters: [25.0, 0.25, 0.04, 3.0] + + - filter: Variable Assignment + assignments: + - name: ObsErrorBound@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorBoundMW@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_latitude: + name: ObsErrorFactorLat@DerivedMetaData + obserr_bound_transmittop: + name: ObsErrorFactorTransmitTop@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_topo: + name: ObsErrorFactorTopo@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + obserr_function: + name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + threhold: 3 + obserr_bound_max: [4.5, 4.5, 3.0, 3.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, + 1.0, 1.0, 2.0, 4.5, 4.5, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0] + + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + function absolute threshold: + - name: ObsErrorBound@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + actions: + - name: set + flag: GrossCheck + ignore: rejected observations + - name: reject + + # Step 11: Inter-Channel Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: InterChannelConsistencyCheck@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use passive_bc: true + sensor: *{{observation_from_jcb}}_sensor_id + use_flag: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 4, 1, 1, 1, 1, 1, 1, + 1] + maxvalue: 1.0e-12 + actions: + - name: set + flag: InterChannelConsistency + ignore: rejected observations + - name: reject + + # Step 12: Useflag Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/ChannelUseflagCheckRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 4, 1, 1, 1, 1, 1, 1, + 1] + minvalue: 1.0e-12 + actions: + - name: set + flag: UseflagCheck + ignore: rejected observations + - name: reject + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/conv_ps.yaml.j2 similarity index 84% rename from parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/conv_ps.yaml.j2 index 640493a7c..ea52885bf 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/conv_ps.yaml.j2 @@ -1,8 +1,9 @@ -- obs space: - name: Surface Pressure - distribution: - name: Halo - halo size: 1250e3 +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: surface_ps obsdatain: engine: type: H5File @@ -15,6 +16,8 @@ max pool size: 1 simulated variables: [stationPressure] + # Observation Operator + # -------------------- obs operator: name: SfcPCorrected variables: @@ -24,13 +27,17 @@ geovar_sfc_geomz: surface_altitude geovar_geomz: geopotential_height + # Linear Observation Operator + # --------------------------- linear obs operator: name: Identity variables: - name: stationPressure + # Observation Prior Filters (QC) + # ------------------------------ obs prior filters: - # Initial Error Assignments for SFC Observations + # Initial Error Assignments for SFC Observations - filter: Perform Action filter variables: - name: stationPressure @@ -45,9 +52,8 @@ round_to_the_nearest_integer: true xvar: name: ObsValue/stationPressure - xvals: [80000, 75000, 70000, 65000, 60000, 55000 ] - errors: [110, 120, 120, 120, 120, 1.0e+11] - + xvals: [80000, 75000, 70000, 65000, 60000, 55000] + errors: [110, 120, 120, 120, 120, 1.0e+11] - filter: Perform Action filter variables: - name: stationPressure @@ -62,10 +68,9 @@ round_to_the_nearest_integer: true xvar: name: ObsValue/stationPressure - xvals: [85000, 80000, 75000, 70000, 65000, 60000, 55000 ] - errors: [ 120, 140, 140, 140, 140, 140, 1.0e+11] - - # Initial Error Assignments for SFCSHIP Observations + xvals: [85000, 80000, 75000, 70000, 65000, 60000, 55000] + errors: [120, 140, 140, 140, 140, 140, 1.0e+11] + # Initial Error Assignments for SFCSHIP Observations - filter: Perform Action filter variables: - name: stationPressure @@ -80,9 +85,8 @@ round_to_the_nearest_integer: true xvar: name: ObsValue/stationPressure - xvals: [60000, 55000 ] - errors: [ 130, 1.0e+11] - + xvals: [60000, 55000] + errors: [130, 1.0e+11] - filter: Perform Action filter variables: - name: stationPressure @@ -108,9 +112,8 @@ round_to_the_nearest_integer: true xvar: name: ObsValue/stationPressure - xvals: [80000, 75000, 70000, 65000, 60000, 55000 ] - errors: [ 110, 120, 120, 120, 120, 1.0e+11] - + xvals: [80000, 75000, 70000, 65000, 60000, 55000] + errors: [110, 120, 120, 120, 120, 1.0e+11] # Create PreQC group variable (pqm in GSI read_prepbufr) - filter: Variable Assignment assignments: @@ -118,7 +121,7 @@ type: float source variable: ObsErrorData/stationPressure - # Set observation quality-realted variables + # Set observation quality-realted variables # Create PreQC group variable (pqm in GSI read_prepbufr) - filter: Variable Assignment assignments: @@ -155,10 +158,10 @@ - filter: Variable Assignment where: - variable: - name: ObsValue/stationPressure + name: ObsValue/stationPressure is_defined: - variable: - name: ObsValue/stationPressure + name: ObsValue/stationPressure maxvalue: 50000.00 where operator: and assignments: @@ -191,28 +194,24 @@ # maxvalue: 106999.0 # action: # name: reject - ## Reject all ObsType 183 #- filter: RejectList # where: # - variable: # name: ObsType/stationPressure # is_in: 183 - - ## Reject surface pressure below 500 hPa + ## Reject surface pressure below 500 hPa #- filter: Bounds Check # filter variables: # - name: stationPressure # minvalue: 50000.00 # action: # name: reject - #- filter: RejectList # where: # - variable: # name: PreQC/stationPressure # is_in: 4-15 - # Inflate obs error based on obs type - filter: Perform Action filter variables: @@ -224,6 +223,8 @@ name: inflate error inflation factor: 1.2 + # Observation Post Filters (QC) + # ----------------------------- obs post filters: # Calculate obs error inflation factors for duplicated observations at the same location - filter: Variable Assignment @@ -237,7 +238,7 @@ variable: stationPressure # Reduce effective observation error based on obs type and subtype - # In this case: reduce effective obs error for buoy + # In this case: reduce effective obs error for buoy - filter: Perform Action filter variables: - name: stationPressure @@ -252,7 +253,7 @@ name: inflate error inflation factor: 0.7 - # Calculate obs error inflation factors for large discrepancies between model and observations + # Calculate obs error inflation factors for large discrepancies between model and observations - filter: Variable Assignment assignments: - name: ObsErrorFactorSfcPressure/stationPressure @@ -285,7 +286,6 @@ - name: ObsValue/stationPressure - name: HofX/stationPressure coefs: [1, -1] - - filter: Variable Assignment assignments: - name: DerivedMetaData/ObsErrorBoundSfcPressure1 @@ -341,7 +341,7 @@ inflation variable: name: ObsErrorFactorDuplicateCheck/stationPressure - # Reject data based on PreUseFlag (usage in GSI) + # Reject data based on PreUseFlag (usage in GSI) - filter: Perform Action filter variables: - name: stationPressure @@ -350,8 +350,14 @@ is_not_in: 0, 1 action: name: reject - # End of Filters - obs localizations: - - localization method: Horizontal Gaspari-Cohn - lengthscale: 1250e3 - max nobs: 10000 + # End of Filters + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 new file mode 100644 index 000000000..547e820d0 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 @@ -0,0 +1,446 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: CRIS-FSR N20 + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [brightness_temperature] + channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} + + # Observation Operator + # -------------------- + obs operator: + name: CRTM + Absorbers: [H2O, O3] + Clouds: [Water, Ice] + Cloud_Fraction: 1.0 + obs options: + Sensor_ID: &{{observation_from_jcb}}_sensor_id cris-fsr_n20 + EndianType: little_endian + CoefficientPath: "{{crtm_coefficient_path}}" + + # Observation Bias Correction (VarBC) + # ----------------------------------- + obs bias: + input file: "{{atm_obsbiasin_path}}/{{atm_obsbiasin_prefix}}{{observation_from_jcb}}{{atm_obsbiasin_suffix}}" + output file: "{{atm_obsbiasout_path}}/{{atm_obsbiasout_prefix}}{{observation_from_jcb}}{{atm_obsbiasout_suffix}}" + variational bc: + predictors: + - name: constant + - name: lapse_rate + order: 2 + tlapse: &{{observation_from_jcb}}_tlapse "{{atm_obsbiasin_path}}/{{atm_obstlapsein_prefix}}{{observation_from_jcb}}{{atm_obstlapsein_suffix}}" + - name: lapse_rate + tlapse: *{{observation_from_jcb}}_tlapse + - name: emissivity + - name: scan_angle + order: 4 + - name: scan_angle + order: 3 + - name: scan_angle + order: 2 + - name: scan_angle + covariance: + minimal required obs number: 20 + variance range: [1.0e-6, 10.0] + step size: 1.0e-4 + largest analysis variance: 10000.0 + prior: + input file: "{{atm_obsbiasin_path}}/{{atm_obsbiascovin_prefix}}{{observation_from_jcb}}{{atm_obsbiascovin_suffix}}" + inflation: + ratio: 1.1 + ratio for small dataset: 2.0 + output file: "{{atm_obsbiasout_path}}/{{atm_obsbiascovout_prefix}}{{observation_from_jcb}}{{atm_obsbiascovout_suffix}}" + # + #obs filters: + ## Wavenumber Check + #- filter: BlackList + # filter variables: + # - name: brightness_temperature + # channels: 1972, 1973, 1974, 1975, 1976, 1977, 1978, 1979, 1980, 1981, + # 1982, 1983, 1984, 1985, 1986, 1987, 2119, 2140, 2143, 2147, + # 2153, 2158, 2161, 2168, 2171, 2175, 2182 + # where: + # - variable: + # name: solar_zenith_angle@MetaData + # maxvalue: 88.9999 + # - variable: + # name: water_area_fraction@GeoVaLs + # minvalue: 1.0e-12 + # action: + # name: reject + #- filter: BlackList + # filter variables: + # - name: brightness_temperature + # channels: *{{observation_from_jcb}}_simulated_channels + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorWavenumIR@ObsFunction + # channels: *{{observation_from_jcb}}_simulated_channels + # options: + # channels: *{{observation_from_jcb}}_simulated_channels + ## Observation Range Sanity Check + #- filter: Bounds Check + # filter variables: + # - name: brightness_temperature + # channels: *{{observation_from_jcb}}_simulated_channels + # minvalue: 50.00001 + # maxvalue: 449.99999 + # action: + # name: reject + ## Topography Check + #- filter: BlackList + # filter variables: + # - name: brightness_temperature + # channels: *{{observation_from_jcb}}_simulated_channels + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorTopoRad@ObsFunction + # channels: *{{observation_from_jcb}}_simulated_channels + # options: + # channels: *{{observation_from_jcb}}_simulated_channels + # sensor: *{{observation_from_jcb}}_sensor_id + ## Transmittance Top Check + #- filter: BlackList + # filter variables: + # - name: brightness_temperature + # channels: *{{observation_from_jcb}}_simulated_channels + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorTransmitTopRad@ObsFunction + # channels: *{{observation_from_jcb}}_simulated_channels + # options: + # channels: *{{observation_from_jcb}}_simulated_channels + ## Cloud Detection Check + #- filter: Bounds Check + # filter variables: + # - name: brightness_temperature + # channels: *{{observation_from_jcb}}_simulated_channels + # test variables: + # - name: CloudDetectMinResidualIR@ObsFunction + # channels: *{{observation_from_jcb}}_simulated_channels + # options: + # channels: *{{observation_from_jcb}}_simulated_channels + # use_flag: [ -1, 1, 1, -1, 1, -1, 1, -1, 1, 1, + # 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, -1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, + # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, + # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, + # -1, 1, -1, -1, 1, -1, -1, -1, 1, -1, + # -1, 1, -1, -1, 1, -1, 1, -1, 1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # 1, 1, -1, -1, 1, -1, -1, -1, 1, 1, + # 1, 1, -1, -1, -1, -1, 1, 1, -1, -1, + # -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, + # -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, + # 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, 1, 1, -1, -1, -1, -1, -1, 1, + # -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, + # 1, -1, -1, -1, -1, -1, -1, 1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1] + # use_flag_clddet: [ -1, 1, -1, -1, -1, -1, 1, -1, 1, -1, + # 1, -1, 1, -1, -1, 1, -1, -1, -1, 1, + # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, + # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, + # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, + # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, + # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, + # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, + # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, + # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, + # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, + # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, + # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, + # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, + # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, + # -1, 1, -1, -1, 1, -1, -1, -1, 1, -1, + # -1, 1, -1, -1, 1, -1, 1, -1, 1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, + # 1, 1, -1, -1, -1, -1, 1, 1, -1, -1, + # -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, + # -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1] + # obserr_dtempf: [0.50,2.00,4.00,2.00,4.00] + # maxvalue: 1.0e-12 + # action: + # name: reject + ## Surface Temperature Jacobian Check over Land + #- filter: Bounds Check + # filter variables: + # - name: brightness_temperature + # channels: *{{observation_from_jcb}}_simulated_channels + # where: + # - variable: + # name: water_area_fraction@GeoVaLs + # maxvalue: 0.99 + # test variables: + # - name: brightness_temperature_jacobian_surface_temperature@ObsDiag + # channels: *{{observation_from_jcb}}_simulated_channels + # maxvalue: 0.2 + ## NSST Retrieval Check + #- filter: Bounds Check + # filter variables: + # - name: brightness_temperature + # channels: *{{observation_from_jcb}}_simulated_channels + # test variables: + # - name: NearSSTRetCheckIR@ObsFunction + # channels: *{{observation_from_jcb}}_simulated_channels + # options: + # channels: *{{observation_from_jcb}}_simulated_channels + # use_flag: [ -1, 1, 1, -1, 1, -1, 1, -1, 1, 1, + # 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, -1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, + # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, + # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, + # -1, 1, -1, -1, 1, -1, -1, -1, 1, -1, + # -1, 1, -1, -1, 1, -1, 1, -1, 1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # 1, 1, -1, -1, 1, -1, -1, -1, 1, 1, + # 1, 1, -1, -1, -1, -1, 1, 1, -1, -1, + # -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, + # -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, + # 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, 1, 1, -1, -1, -1, -1, -1, 1, + # -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, + # 1, -1, -1, -1, -1, -1, -1, 1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1 ] + # obserr_demisf: [0.01,0.02,0.03,0.02,0.03] + # obserr_dtempf: [0.50,2.00,4.00,2.00,4.00] + # maxvalue: 1.0e-12 + # action: + # name: reject + ## Surface Jacobians Check + #- filter: BlackList + # filter variables: + # - name: brightness_temperature + # channels: *{{observation_from_jcb}}_simulated_channels + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorSurfJacobianRad@ObsFunction + # channels: *{{observation_from_jcb}}_simulated_channels + # options: + # channels: *{{observation_from_jcb}}_simulated_channels + # obserr_demisf: [0.01, 0.02, 0.03, 0.02, 0.03] + # obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] + ## Gross check + #- filter: Background Check + # filter variables: + # - name: brightness_temperature + # channels: *{{observation_from_jcb}}_simulated_channels + # function absolute threshold: + # - name: ObsErrorBoundIR@ObsFunction + # channels: *{{observation_from_jcb}}_simulated_channels + # options: + # channels: *{{observation_from_jcb}}_simulated_channels + # obserr_bound_latitude: + # name: ObsErrorFactorLatRad@ObsFunction + # options: + # latitude_parameters: [25.0, 0.5, 0.04, 1.0] + # obserr_bound_transmittop: + # name: ObsErrorFactorTransmitTopRad@ObsFunction + # channels: *{{observation_from_jcb}}_simulated_channels + # options: + # channels: *{{observation_from_jcb}}_simulated_channels + # obserr_bound_max: [ 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0 ] + # action: + # name: reject + ## Useflag Check + #- filter: Bounds Check + # filter variables: + # - name: brightness_temperature + # channels: *{{observation_from_jcb}}_simulated_channels + # test variables: + # - name: ChannelUseflagCheckRad@ObsFunction + # channels: *{{observation_from_jcb}}_simulated_channels + # options: + # channels: *{{observation_from_jcb}}_simulated_channels + # use_flag: [ -1, 1, 1, -1, 1, -1, 1, -1, 1, 1, + # 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, -1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, + # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, + # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, + # -1, 1, -1, -1, 1, -1, -1, -1, 1, -1, + # -1, 1, -1, -1, 1, -1, 1, -1, 1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # 1, 1, -1, -1, 1, -1, -1, -1, 1, 1, + # 1, 1, -1, -1, -1, -1, 1, 1, -1, -1, + # -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, + # -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, + # 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, 1, 1, -1, -1, -1, -1, -1, 1, + # -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, + # 1, -1, -1, -1, -1, -1, -1, 1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1 ] + # minvalue: 1.0e-12 + # action: + # name: reject + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/cris-fsr_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/cris-fsr_npp.yaml.j2 new file mode 100644 index 000000000..90194caf8 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/cris-fsr_npp.yaml.j2 @@ -0,0 +1,445 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: CRIS-FSR NPP + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [brightness_temperature] + channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} + + # Observation Operator + # -------------------- + obs operator: + name: CRTM + Absorbers: [H2O, O3] + Clouds: [Water, Ice] + Cloud_Fraction: 1.0 + obs options: + Sensor_ID: &{{observation_from_jcb}}_sensor_id cris-fsr_npp + EndianType: little_endian + CoefficientPath: "{{crtm_coefficient_path}}" + + # Observation Bias Correction (VarBC) + # ----------------------------------- + obs bias: + input file: "{{atm_obsbiasin_path}}/{{atm_obsbiasin_prefix}}{{observation_from_jcb}}{{atm_obsbiasin_suffix}}" + output file: "{{atm_obsbiasout_path}}/{{atm_obsbiasout_prefix}}{{observation_from_jcb}}{{atm_obsbiasout_suffix}}" + variational bc: + predictors: + - name: constant + - name: lapse_rate + order: 2 + tlapse: &{{observation_from_jcb}}_tlapse "{{atm_obsbiasin_path}}/{{atm_obstlapsein_prefix}}{{observation_from_jcb}}{{atm_obstlapsein_suffix}}" + - name: lapse_rate + tlapse: *{{observation_from_jcb}}_tlapse + - name: emissivity + - name: scan_angle + order: 4 + - name: scan_angle + order: 3 + - name: scan_angle + order: 2 + - name: scan_angle + covariance: + minimal required obs number: 20 + variance range: [1.0e-6, 10.0] + step size: 1.0e-4 + largest analysis variance: 10000.0 + prior: + input file: "{{atm_obsbiasin_path}}/{{atm_obsbiascovin_prefix}}{{observation_from_jcb}}{{atm_obsbiascovin_suffix}}" + inflation: + ratio: 1.1 + ratio for small dataset: 2.0 + output file: "{{atm_obsbiasout_path}}/{{atm_obsbiascovout_prefix}}{{observation_from_jcb}}{{atm_obsbiascovout_suffix}}" + #obs filters: + ## Wavenumber Check + #- filter: BlackList + # filter variables: + # - name: brightness_temperature + # channels: 1972, 1973, 1974, 1975, 1976, 1977, 1978, 1979, 1980, 1981, + # 1982, 1983, 1984, 1985, 1986, 1987, 2119, 2140, 2143, 2147, + # 2153, 2158, 2161, 2168, 2171, 2175, 2182 + # where: + # - variable: + # name: solar_zenith_angle@MetaData + # maxvalue: 88.9999 + # - variable: + # name: water_area_fraction@GeoVaLs + # minvalue: 1.0e-12 + # action: + # name: reject + #- filter: BlackList + # filter variables: + # - name: brightness_temperature + # channels: *{{observation_from_jcb}}_simulated_channels + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorWavenumIR@ObsFunction + # channels: *{{observation_from_jcb}}_simulated_channels + # options: + # channels: *{{observation_from_jcb}}_simulated_channels + ## Observation Range Sanity Check + #- filter: Bounds Check + # filter variables: + # - name: brightness_temperature + # channels: *{{observation_from_jcb}}_simulated_channels + # minvalue: 50.00001 + # maxvalue: 449.99999 + # action: + # name: reject + ## Topography Check + #- filter: BlackList + # filter variables: + # - name: brightness_temperature + # channels: *{{observation_from_jcb}}_simulated_channels + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorTopoRad@ObsFunction + # channels: *{{observation_from_jcb}}_simulated_channels + # options: + # channels: *{{observation_from_jcb}}_simulated_channels + # sensor: *{{observation_from_jcb}}_sensor_id + ## Transmittance Top Check + #- filter: BlackList + # filter variables: + # - name: brightness_temperature + # channels: *{{observation_from_jcb}}_simulated_channels + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorTransmitTopRad@ObsFunction + # channels: *{{observation_from_jcb}}_simulated_channels + # options: + # channels: *{{observation_from_jcb}}_simulated_channels + ## Cloud Detection Check + #- filter: Bounds Check + # filter variables: + # - name: brightness_temperature + # channels: *{{observation_from_jcb}}_simulated_channels + # test variables: + # - name: CloudDetectMinResidualIR@ObsFunction + # channels: *{{observation_from_jcb}}_simulated_channels + # options: + # channels: *{{observation_from_jcb}}_simulated_channels + # use_flag: [ -1, 1, 1, -1, 1, -1, 1, -1, 1, 1, + # 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, -1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, + # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, + # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, + # -1, 1, -1, -1, 1, -1, -1, -1, 1, -1, + # -1, 1, -1, -1, 1, -1, 1, -1, 1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # 1, 1, -1, -1, 1, -1, -1, -1, 1, 1, + # 1, 1, -1, -1, -1, -1, 1, 1, -1, -1, + # -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, + # -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, + # 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, 1, 1, -1, -1, -1, -1, -1, 1, + # -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, + # 1, -1, -1, -1, -1, -1, -1, 1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1] + # use_flag_clddet: [ -1, 1, -1, -1, -1, -1, 1, -1, 1, -1, + # 1, -1, 1, -1, -1, 1, -1, -1, -1, 1, + # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, + # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, + # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, + # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, + # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, + # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, + # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, + # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, + # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, + # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, + # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, + # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, + # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, + # -1, 1, -1, -1, 1, -1, -1, -1, 1, -1, + # -1, 1, -1, -1, 1, -1, 1, -1, 1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, + # 1, 1, -1, -1, -1, -1, 1, 1, -1, -1, + # -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, + # -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1] + # obserr_dtempf: [0.50,2.00,4.00,2.00,4.00] + # maxvalue: 1.0e-12 + # action: + # name: reject + ## Surface Temperature Jacobian Check over Land + #- filter: Bounds Check + # filter variables: + # - name: brightness_temperature + # channels: *{{observation_from_jcb}}_simulated_channels + # where: + # - variable: + # name: water_area_fraction@GeoVaLs + # maxvalue: 0.99 + # test variables: + # - name: brightness_temperature_jacobian_surface_temperature@ObsDiag + # channels: *{{observation_from_jcb}}_simulated_channels + # maxvalue: 0.2 + ## NSST Retrieval Check + #- filter: Bounds Check + # filter variables: + # - name: brightness_temperature + # channels: *{{observation_from_jcb}}_simulated_channels + # test variables: + # - name: NearSSTRetCheckIR@ObsFunction + # channels: *{{observation_from_jcb}}_simulated_channels + # options: + # channels: *{{observation_from_jcb}}_simulated_channels + # use_flag: [ -1, 1, 1, -1, 1, -1, 1, -1, 1, 1, + # 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, -1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, + # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, + # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, + # -1, 1, -1, -1, 1, -1, -1, -1, 1, -1, + # -1, 1, -1, -1, 1, -1, 1, -1, 1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # 1, 1, -1, -1, 1, -1, -1, -1, 1, 1, + # 1, 1, -1, -1, -1, -1, 1, 1, -1, -1, + # -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, + # -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, + # 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, 1, 1, -1, -1, -1, -1, -1, 1, + # -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, + # 1, -1, -1, -1, -1, -1, -1, 1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1 ] + # obserr_demisf: [0.01,0.02,0.03,0.02,0.03] + # obserr_dtempf: [0.50,2.00,4.00,2.00,4.00] + # maxvalue: 1.0e-12 + # action: + # name: reject + ## Surface Jacobians Check + #- filter: BlackList + # filter variables: + # - name: brightness_temperature + # channels: *{{observation_from_jcb}}_simulated_channels + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorSurfJacobianRad@ObsFunction + # channels: *{{observation_from_jcb}}_simulated_channels + # options: + # channels: *{{observation_from_jcb}}_simulated_channels + # obserr_demisf: [0.01, 0.02, 0.03, 0.02, 0.03] + # obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] + ## Gross check + #- filter: Background Check + # filter variables: + # - name: brightness_temperature + # channels: *{{observation_from_jcb}}_simulated_channels + # function absolute threshold: + # - name: ObsErrorBoundIR@ObsFunction + # channels: *{{observation_from_jcb}}_simulated_channels + # options: + # channels: *{{observation_from_jcb}}_simulated_channels + # obserr_bound_latitude: + # name: ObsErrorFactorLatRad@ObsFunction + # options: + # latitude_parameters: [25.0, 0.5, 0.04, 1.0] + # obserr_bound_transmittop: + # name: ObsErrorFactorTransmitTopRad@ObsFunction + # channels: *{{observation_from_jcb}}_simulated_channels + # options: + # channels: *{{observation_from_jcb}}_simulated_channels + # obserr_bound_max: [ 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + # 2.0 ] + # action: + # name: reject + ## Useflag Check + #- filter: Bounds Check + # filter variables: + # - name: brightness_temperature + # channels: *{{observation_from_jcb}}_simulated_channels + # test variables: + # - name: ChannelUseflagCheckRad@ObsFunction + # channels: *{{observation_from_jcb}}_simulated_channels + # options: + # channels: *{{observation_from_jcb}}_simulated_channels + # use_flag: [ -1, 1, 1, -1, 1, -1, 1, -1, 1, 1, + # 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, -1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, + # -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, + # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, + # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, + # -1, 1, -1, -1, 1, -1, -1, -1, 1, -1, + # -1, 1, -1, -1, 1, -1, 1, -1, 1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # 1, 1, -1, -1, 1, -1, -1, -1, 1, 1, + # 1, 1, -1, -1, -1, -1, 1, 1, -1, -1, + # -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, + # -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, + # 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, 1, 1, -1, -1, -1, -1, -1, 1, + # -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, + # 1, -1, -1, -1, -1, -1, -1, 1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + # -1 ] + # minvalue: 1.0e-12 + # action: + # name: reject + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro.yaml.j2 new file mode 100644 index 000000000..3a984c0c2 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/gnssro.yaml.j2 @@ -0,0 +1,157 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: gnssrobndnbam + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + # obsgrouping: + # group variables: [ "sequenceNumber" ] + # sort variable: "impactHeightRO" + # sort order: "ascending" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + simulated variables: [bendingAngle] + + # Observation Operator + # -------------------- + obs operator: + name: GnssroBndNBAM + obs options: + use_compress: 1 + sr_steps: 2 + vertlayer: full + super_ref_qc: NBAM + + # Observation Filters (QC) + # ------------------------ + obs filters: + # Apply gross check using pccf + # Step 0-A: Create Diagnostic Flags + # Diagnostic flag for pccf + - filter: Create Diagnostic Flags + filter variables: + - name: bendingAngle + flags: + - name: pccfCheckReject + initial value: false + force reinitialization: true + + # Diagnostic flag for qfro + - filter: Create Diagnostic Flags + filter variables: + - name: bendingAngle + flags: + - name: qfroCheckReject + initial value: false + force reinitialization: true + + # Step 0-B: pccf Check - good: 0.1-100, reject: 0 + - filter: Bounds Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/satelliteIdentifier + is_in: 265-269,750-755,44,786,820,825 + test variables: + - name: MetaData/pccf + minvalue: 0.1 + maxvalue: 100.1 + actions: + - name: set + flag: pccfCheckReject + - name: reject + + # Step 0-B: qfro Check - good: 0, reject: 1 + - filter: Bounds Check + - filter: RejectList + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/satelliteIdentifier + is_in: 3-5,421,440,821 + test variables: + - name: MetaData/qualityFlags + minvalue: -0.1 + maxvalue: 0.1 + actions: + - name: set + flag: qfroCheckReject + - name: reject + + #1. gpstop + - filter: Domain Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/impactHeightRO + minvalue: 0 + maxvalue: 55000.1 + action: + name: reject + #2. commgpstop + - filter: Bounds Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/satelliteIdentifier + is_in: 265,266,267,268,269 + test variables: + - name: MetaData/impactHeightRO + maxvalue: 45000.1 + action: + name: reject + #3. metop below 8 km + - filter: Bounds Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/satelliteIdentifier + is_in: 3-5 + test variables: + - name: MetaData/impactHeightRO + minvalue: 8000.1 + action: + name: reject + #4. assign obs error + - filter: ROobserror + filter variables: + - name: bendingAngle + errmodel: NBAM + #5. RONBAM cut off check + - filter: Background Check RONBAM + filter variables: + - name: bendingAngle + #6. Obs error inflate + - filter: Background Check RONBAM + filter variables: + - name: bendingAngle + action: + name: RONBAMErrInflate + #7. Background check + #- filter: Background Check + # filter variables: + # - name: bendingAngle + # threshold: 10 + # action: + # name: reject + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 new file mode 100644 index 000000000..b546ac5b3 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 @@ -0,0 +1,579 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: IASI METOP-A + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + observed variables: [radiance] + simulated variables: [brightnessTemperature] + derived variables: [brightnessTemperature] + channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} + + # Observation Operator + # -------------------- + obs operator: + name: CRTM + Absorbers: [H2O, O3, CO2] + obs options: + Sensor_ID: &{{observation_from_jcb}}_sensor_id iasi_metop-a + EndianType: little_endian + CoefficientPath: "{{crtm_coefficient_path}}" + linear obs operator: + Absorbers: [H2O, O3] + + # Observation Bias Correction (VarBC) + # ----------------------------------- + obs bias: + input file: "{{atm_obsbiasin_path}}/{{atm_obsbiasin_prefix}}{{observation_from_jcb}}{{atm_obsbiasin_suffix}}" + output file: "{{atm_obsbiasout_path}}/{{atm_obsbiasout_prefix}}{{observation_from_jcb}}{{atm_obsbiasout_suffix}}" + variational bc: + predictors: + - name: constant + - name: lapse_rate + order: 2 + tlapse: &{{observation_from_jcb}}_tlapse "{{atm_obsbiasin_path}}/{{atm_obstlapsein_prefix}}{{observation_from_jcb}}{{atm_obstlapsein_suffix}}" + - name: lapse_rate + tlapse: *{{observation_from_jcb}}_tlapse + - name: emissivity + - name: scan_angle + order: 4 + - name: scan_angle + order: 3 + - name: scan_angle + order: 2 + - name: scan_angle + covariance: + minimal required obs number: 20 + variance range: [1.0e-6, 10.0] + step size: 1.0e-4 + largest analysis variance: 10000.0 + prior: + input file: "{{atm_obsbiasin_path}}/{{atm_obsbiascovin_prefix}}{{observation_from_jcb}}{{atm_obsbiascovin_suffix}}" + inflation: + ratio: 1.1 + ratio for small dataset: 2.0 + output file: "{{atm_obsbiasout_path}}/{{atm_obsbiascovout_prefix}}{{observation_from_jcb}}{{atm_obsbiascovout_suffix}}" + + # Observation Pre Filters (QC) + # ---------------------------- + obs pre filters: + # Step 0-A: Create Diagnostic Flags + #- filter: Create Diagnostic Flags + # filter variables: + # - name: brightnessTemperature + # channels: *{{observation_from_jcb}}_simulated_channels + # flags: + # - name: ScanEdgeRemoval + # initial value: false + # force reinitialization: false + # - name: Thinning + # initial value: false + # force reinitialization: false + # - name: ShortwaveIRCheck + # initial value: false + # force reinitialization: false + # - name: ObsValueRangeCheck + # initial value: false + # force reinitialization: false + # - name: CloudDetection + # initial value: false + # force reinitialization: false + # - name: NSSTCheck + # initial value: false + # force reinitialization: false + # - name: GrossCheck + # initial value: false + # force reinitialization: false + # - name: InterChannelConsistency + # initial value: false + # force reinitialization: false + # - name: UseFlagCheck + # initial value: false + # force reinitialization: false + # Step 0-B: Create Derived Variables + # Assign channel wavenumbers in m-1 + - filter: Variable Assignment + assignments: + - name: MetaData/sensorCentralWavenumber + type: float + channels: *{{observation_from_jcb}}_simulated_channels + function: + name: ObsFunction/LinearCombination + options: + variables: + - name: ObsValue/radiance + channels: *{{observation_from_jcb}}_simulated_channels + coefs: [25.0] + intercept: 64475.0 + use channel numbers: true + + # Transform radiance to brightness temperature + - filter: Variable Transforms + Transform: SatBrightnessTempFromRad + transform from: + name: ObsValue/radiance + channels: *{{observation_from_jcb}}_simulated_channels + spectral variable: + name: MetaData/sensorCentralWavenumber + channels: *{{observation_from_jcb}}_simulated_channels + radiance units: wavenumber + planck1: 1.191042953e-16 + planck2: 1.4387774e-2 + + # Step 0-C: Assign Observation Error + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: assign error + error parameter vector: &{{observation_from_jcb}}_oberr [1.38, 0.81, 0.75, 0.79, 0.72, 0.74, + 0.68, 0.72, 0.65, 0.65, 0.65, 0.69, 0.64, 0.64, 0.65, 0.67, 0.62, 0.61, 0.62, + 0.64, 0.59, 0.76, 1.22, 0.78, 0.64, 0.62, 0.61, 0.69, 0.65, 0.59, 0.61, 0.59, + 0.68, 0.62, 0.68, 4.38, 3.05, 2.31, 1.56, 1.33, 1.58, 0.93, 1.67, 0.72, 0.57, + 0.58, 0.55, 0.68, 0.59, 0.68, 0.59, 0.65, 0.58, 0.62, 0.64, 0.58, 0.64, 0.55, + 0.64, 0.5, 0.82, 0.59, 0.62, 0.51, 0.64, 0.52, 0.51, 0.51, 0.76, 0.52, 0.57, + 0.55, 0.69, 0.58, 0.65, 0.61, 0.59, 0.64, 0.76, 0.72, 1.05, 0.75, 0.51, 0.65, + 1.3, 0.69, 0.93, 1.49, 1.12, 0.68, 0.66, 0.67, 0.59, 0.59, 0.69, 0.67, 0.64, + 0.62, 0.72, 0.69, 0.66, 0.79, 0.78, 0.74, 0.88, 0.77, 0.88, 0.86, 1, 0.87, + 0.85, 0.88, 0.84, 0.84, 0.84, 0.8, 0.8, 0.87, 0.98, 0.52, 0.65, 0.69, 0.61, + 0.6, 0.67, 0.79, 0.62, 0.66, 0.7, 0.65, 0.62, 0.61, 0.62, 0.53, 0.6, 0.68, + 0.95, 0.63, 0.97, 0.65, 0.98, 0.58, 0.73, 0.65, 0.85, 0.99, 0.76, 0.85, 0.97, + 0.77, 0.62, 0.63, 1.21, 1.41, 1.55, 1.78, 1.35, 1.14, 1.69, 1.79, 1.46, 1.63, + 1.94, 2.01, 1.24, 1.76, 1.26, 1.47, 1.9, 1.66, 2.13, 1.49, 1.52, 1.55, 1.96, + 2.31, 2.33, 2.32, 2.31, 2.33, 2.23, 2.33, 1.84, 2.29, 2.28, 2.28, 2.28, 2.26, + 2.26, 2.26, 2.27, 2.24, 2.23, 2.24, 2.26, 2.28, 2.28, 2.3, 2.15, 2.31, 2.37, + 2.27, 2.29, 2.29, 2.23, 2.28, 2.32, 2.32, 2.31, 2.32, 2.32, 2.31, 2.31, 2.28, + 2.29, 2.28, 2.26, 2.29, 2.27, 2.26, 2.25, 2.27, 2.24, 2.21, 2.24, 2.17, 2.18, + 2.17, 2.21, 1.99, 2.16, 2.2, 2.13, 2.12, 2.13, 2.1, 2.12, 2.11, 2.09, 2.09, + 2.08, 2.09, 2.04, 2.04, 2.1, 2.01, 2.05, 2.03, 2.06, 1.98, 1.95, 1.94, 1.91, + 1.7, 1.76, 1.77, 1.83, 2.04, 1.91, 1.99, 1.99, 2.07, 2.02, 2.04, 2.1, 2.06, + 2.18, 2.21, 2.24, 2.23, 2.23, 1.98, 2.2, 2.18, 2.18, 2.21, 2.23, 2.24, 2.24, + 2.25, 1.8, 2.24, 1.73, 1.73, 2.27, 1.67, 2.21, 1.72, 2.23, 2.23, 2.23, 2.24, + 2.23, 2.12, 2.17, 1.74, 2.02, 1.88, 1.67, 1.73, 1.83, 1.82, 1.73, 1.83, 2.19, + 1.84, 1.89, 1.6, 1.71, 1.86, 1.85, 1.84, 1.87, 1.91, 1.52, 1.95, 1.87, 1.89, + 1.91, 1.91, 1.93, 1.9, 1.91, 1.9, 1.89, 1.89, 1.91, 1.9, 1.91, 1.91, 1.91, + 1.93, 1.94, 1.91, 1.92, 1.77, 1.91, 1.95, 1.19, 1.96, 1.98, 1.94, 1.55, 1.91, + 1.92, 1.92, 1.97, 1.93, 1.99, 1.86, 1.12, 1.93, 1.92, 1.95, 1.85, 1.84, 1.91, + 1.12, 1.82, 1.82, 1.95, 1.24, 1.94, 1.96, 1.21, 1.83, 1.96, 1.36, 1.96, 1.82, + 1.92, 1.68, 1.93, 1.23, 1.96, 1.93, 1.86, 1.41, 1.16, 1.6, 1.25, 1.2, 1.65, + 1.66, 1.87, 1.94, 1.96, 1.91, 1.25, 1.93, 1.91, 1.7, 0.99, 1.81, 1.92, 1.95, + 1.5, 1.47, 1.15, 1.58, 1.18, 1.82, 1.13, 1.83, 1.91, 1.26, 1.27, 1.91, 1.45, + 1.6, 1.29, 1.94, 1.94, 1.23, 1.95, 1.21, 1.94, 1.86, 1.9, 1.33, 1.75, 2.02, + 1.98, 2.03, 1.83, 1.5, 2.04, 2.02, 1.9, 2, 2.02, 1.95, 1.93, 1.95, 1.95, 1.99, + 2, 1.94, 1.96, 1.86, 1.92, 1.88, 1.86, 1.84, 1.87, 1.77, 1.89, 1.89, 1.88, + 1.94, 1.82, 1.79, 1.86, 2.06, 2.33, 1.88, 1.86, 1.81, 1.8, 1.8, 1.86, 1.9, + 2, 2.06, 2.1, 2.2, 2, 2.16, 1.98, 1.8, 1.8, 1.85, 1.75, 2.04, 2.19, 2.14, + 2.19, 1.86, 2.1, 2.11, 2.18, 2.03, 2.28, 2.19, 2.26, 2.26, 2.21, 2.21, 2.26, + 2.33, 2.27, 2.21, 2.12, 2.23, 2.26, 2.25, 1.88, 2.26, 2.24, 2.36, 2.29, 2.35, + 2.3, 2.27, 2.08, 2.05, 2.27, 2.28, 2.27, 2.28, 1.97, 2.25, 2.25, 2.25, 2.31, + 2.28, 2.27, 2.13, 2.24, 2.28, 2.28, 2.41, 2.34, 9.32, 2.28, 2.38, 2.27, 2.27, + 2.39, 2.11, 2.09, 2.1, 2.06, 2.12, 2.08, 2, 1.93, 2.02, 2.55, 1.54, 1.64, + 1.51, 1.55, 2.82, 2.92, 2.55, 2.37, 1.85, 1.6, 1.72, 1.74, 1.79, 1.9, 1.94, + 2, 2.04, 2.08, 2.12, 2.13, 2.16, 2.18, 2.18, 2.2, 2.2, 2.41, 2.39, 2.38, 2.4, + 2.42, 2.41, 2.43, 2.45, 2.43, 2.45, 2.43, 2.4, 2.44, 2.4, 2.42, 2.43, 2.45, + 2.45, 2.45, 2.46, 2.45, 2.45, 2.43, 2.51, 2.48, 2.48, 2.53, 2.46, 2.49, 2.5, + 2.5, 2.5, 2.52, 2.52, 2.54, 2.5, 2.48, 2.5, 2.55, 2.5, 2.48, 2.5, 2.5, 2.52, + 2.52, 2.48, 2.5, 2.5, 2.52, 2.46, 2.53, 9] + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Step 1: Remove Observations from the Edge of the Scan + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: MetaData/sensorScanPosition + is_in: 5-56 + action: + name: reject + # actions: + # - name: set + # flag: ScanEdgeRemoval + # - name: reject + # Step 2: Data Thinning + - filter: Gaussian Thinning + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + horizontal_mesh: 145 + use_reduced_horizontal_grid: true + distance_norm: geodesic + priority_variable: MetaData/fractionOfClearPixelsInFOV + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + action: + name: reject + # actions: + # - name: set + # flag: Thinning + # - name: reject + # Step 3: Wavenumber Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: 7024, 7027, 7029, 7032, 7038, 7043, 7046, 7049, 7069, 7072, 7076, + 7081, 7084, 7089, 7099, 7209, 7222, 7231, 7235, 7247, 7267, 7269, 7284, 7389, + 7419, 7423, 7424, 7426, 7428, 7431, 7436, 7444, 7475, 7549, 7584, 7665, 7666, + 7831, 7836, 7853, 7865, 7885, 7888, 7912, 7950, 7972, 7980, 7995, 8007, 8015, + 8055, 8078 + where: + - variable: + name: MetaData/solarZenithAngle + maxvalue: 88.9999 + - variable: + name: water_area_fraction@GeoVaLs + minvalue: 1.0e-12 + action: + name: reject + # actions: + # - name: set + # flag: ShortwaveIRCheck + # - name: reject + # Step 4: Observation Error Inflation based on Wavenumber + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsErrorFactorWavenumIR@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + # Step 5: Observation Range Sanity Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + minvalue: 50.00001 + maxvalue: 449.99999 + # maxvalue: 100.00000 + action: + name: reject + # actions: + # - name: set + # flag: ObsValueRangeCheck + # - name: reject + # Step 6: Topography Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorTopoRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + sensor: *{{observation_from_jcb}}_sensor_id + + # Step 7: Transmittance Top Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorTransmitTopRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + # Step 8: Cloud Detection Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/CloudDetectMinResidualIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: [1, -1, -1, -1, 1, -1, -1, -1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, + 1, 1, -1, 1, 1, 1, -1, -1, 1, 1, -1, 1, -1, 1, -1, 1, -1, -1, -1, -1, -1, + -1, -1, -1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, -1, + 1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, -1, -1, + 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, 1, 1, 1, + 1, 1, 1, -1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, 1, 1, -1, -1, -1, + 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, + 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, -1, 1, 1, 1, -1, 1, 1, -1, 1, -1, + 1, -1, -1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, -1, -1, -1, 1, 1, 1, -1, + -1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, 1, -1, 1, 1, -1, 1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, + -1, -1, 1, -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, 1, + -1, 1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, -1, -1, -1, -1, -1, 1, -1, 1, -1, + 1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, + -1, -1, 1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, + -1, -1, -1, -1, -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, 1, + -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1] + use_flag_clddet: [1, -1, -1, -1, 1, -1, -1, -1, 1, -1, 1, -1, 1, -1, 1, 1, + 1, -1, 1, 1, -1, 1, 1, 1, -1, -1, 1, 1, -1, 1, -1, 1, -1, 1, -1, -1, -1, + -1, -1, -1, -1, -1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, + 1, 1, -1, 1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, + -1, -1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, 1, + 1, 1, 1, 1, 1, -1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, 1, 1, -1, + -1, -1, 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, 1, + 1, 1, 1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, -1, 1, 1, 1, -1, 1, 1, + -1, 1, -1, 1, -1, -1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, -1, -1, -1, 1, + 1, 1, -1, -1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, 1, -1, 1, 1, -1, 1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 1, -1, 1, -1, 1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, + -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1] + obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] + error parameter vector: *{{observation_from_jcb}}_oberr + maxvalue: 1.0e-12 + action: + name: reject + # actions: + # - name: set + # flag: CloudDetection + # - name: reject + # Step 9: NSST Retrieval Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: NearSSTRetCheckIR@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: [1, -1, -1, -1, 1, -1, -1, -1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, + 1, 1, -1, 1, 1, 1, -1, -1, 1, 1, -1, 1, -1, 1, -1, 1, -1, -1, -1, -1, -1, + -1, -1, -1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, -1, + 1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, -1, -1, + 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, 1, 1, 1, + 1, 1, 1, -1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, 1, 1, -1, -1, -1, + 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, + 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, -1, 1, 1, 1, -1, 1, 1, -1, 1, -1, + 1, -1, -1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, -1, -1, -1, 1, 1, 1, -1, + -1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, 1, -1, 1, 1, -1, 1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, + -1, -1, 1, -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, 1, + -1, 1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, -1, -1, -1, -1, -1, 1, -1, 1, -1, + 1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, + -1, -1, 1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, + -1, -1, -1, -1, -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, 1, + -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1] + obserr_demisf: [0.01, 0.02, 0.03, 0.02, 0.03] + obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] + maxvalue: 1.0e-12 + action: + name: reject + # actions: + # - name: set + # flag: NSSTCheck + # - name: reject + # Step 10: Surface Jacobians Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorSurfJacobianRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_demisf: [0.01, 0.02, 0.03, 0.02, 0.03] + obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] + sensor: *{{observation_from_jcb}}_sensor_id + + # Step 11: Gross check + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + function absolute threshold: + - name: ObsFunction/ObsErrorBoundIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_latitude: + name: ObsFunction/ObsErrorFactorLatRad + options: + latitude_parameters: [25.0, 0.5, 0.04, 1.0] + obserr_bound_transmittop: + name: ObsFunction/ObsErrorFactorTransmitTopRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_max: [3.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 4.0, 2.0, 2.0, 2.0, + 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 4.0, 4.0, 3.5, + 2.5, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 3.5, 2.0, 2.0, 2.0, 2.0, 2.0, + 2.0, 3.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + 2.0, 2.0, 2.0, 2.5, 2.0, 2.5, 2.5, 3.0, 2.5, 2.5, 2.5, 2.5, 3.5, 2.5, 2.5, + 3.0, 3.5, 3.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.5, 4.5, 4.5, 4.5, 4.5, 4.0, + 4.5, 4.0, 4.0, 4.5, 2.5, 3.0, 2.5, 3.0, 2.5, 3.0, 2.0, 2.5, 2.5, 3.0, 3.0, + 2.5, 3.0, 3.0, 3.0, 2.5, 2.5, 4.0, 4.5, 4.5, 5.0, 4.0, 4.0, 5.0, 5.0, 5.0, + 5.0, 5.5, 5.5, 4.0, 5.0, 4.0, 4.5, 5.5, 5.5, 6.0, 4.5, 4.5, 4.0, 5.0, 5.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 5.5, 4.5, 6.0, 5.0, 5.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 5.0, 6.0, 6.0, 6.0, 4.0, 6.0, 6.0, 6.0, 6.0, 4.5, 6.0, 6.0, 4.5, + 6.0, 6.0, 6.0, 6.0, 6.0, 5.0, 6.0, 6.0, 6.0, 5.0, 6.0, 6.0, 5.0, 6.0, 5.0, + 6.0, 6.0, 6.0, 5.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0] + error parameter vector: *{{observation_from_jcb}}_oberr + action: + name: reject + # actions: + # - name: set + # flag: GrossCheck + # - name: reject + # Step 12: Useflag Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/ChannelUseflagCheckRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: [1, -1, -1, -1, 1, -1, -1, -1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, + 1, 1, -1, 1, 1, 1, -1, -1, 1, 1, -1, 1, -1, 1, -1, 1, -1, -1, -1, -1, -1, + -1, -1, -1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, -1, + 1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, -1, -1, + 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, 1, 1, 1, + 1, 1, 1, -1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, 1, 1, -1, -1, -1, + 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, + 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, -1, 1, 1, 1, -1, 1, 1, -1, 1, -1, + 1, -1, -1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, -1, -1, -1, 1, 1, 1, -1, + -1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, 1, -1, 1, 1, -1, 1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, + -1, -1, 1, -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, 1, + -1, 1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, -1, -1, -1, -1, -1, 1, -1, 1, -1, + 1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, + -1, -1, 1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, + -1, -1, -1, -1, -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, 1, + -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1] + minvalue: 1.0e-12 + action: + name: reject + # actions: + # - name: set + # flag: UseFlagCheck + # - name: reject + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 new file mode 100644 index 000000000..51696bd96 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 @@ -0,0 +1,579 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: IASI METOP-B + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + observed variables: [radiance] + simulated variables: [brightnessTemperature] + derived variables: [brightnessTemperature] + channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} + + # Observation Operator + # -------------------- + obs operator: + name: CRTM + Absorbers: [H2O, O3, CO2] + obs options: + Sensor_ID: &{{observation_from_jcb}}_sensor_id iasi_metop-b + EndianType: little_endian + CoefficientPath: "{{crtm_coefficient_path}}" + linear obs operator: + Absorbers: [H2O, O3] + + # Observation Bias Correction (VarBC) + # ----------------------------------- + obs bias: + input file: "{{atm_obsbiasin_path}}/{{atm_obsbiasin_prefix}}{{observation_from_jcb}}{{atm_obsbiasin_suffix}}" + output file: "{{atm_obsbiasout_path}}/{{atm_obsbiasout_prefix}}{{observation_from_jcb}}{{atm_obsbiasout_suffix}}" + variational bc: + predictors: + - name: constant + - name: lapse_rate + order: 2 + tlapse: &{{observation_from_jcb}}_tlapse "{{atm_obsbiasin_path}}/{{atm_obstlapsein_prefix}}{{observation_from_jcb}}{{atm_obstlapsein_suffix}}" + - name: lapse_rate + tlapse: *{{observation_from_jcb}}_tlapse + - name: emissivity + - name: scan_angle + order: 4 + - name: scan_angle + order: 3 + - name: scan_angle + order: 2 + - name: scan_angle + covariance: + minimal required obs number: 20 + variance range: [1.0e-6, 10.0] + step size: 1.0e-4 + largest analysis variance: 10000.0 + prior: + input file: "{{atm_obsbiasin_path}}/{{atm_obsbiascovin_prefix}}{{observation_from_jcb}}{{atm_obsbiascovin_suffix}}" + inflation: + ratio: 1.1 + ratio for small dataset: 2.0 + output file: "{{atm_obsbiasout_path}}/{{atm_obsbiascovout_prefix}}{{observation_from_jcb}}{{atm_obsbiascovout_suffix}}" + + # Observation Pre Filters (QC) + # ---------------------------- + obs pre filters: + # Step 0-A: Create Diagnostic Flags + #- filter: Create Diagnostic Flags + # filter variables: + # - name: brightnessTemperature + # channels: *{{observation_from_jcb}}_simulated_channels + # flags: + # - name: ScanEdgeRemoval + # initial value: false + # force reinitialization: false + # - name: Thinning + # initial value: false + # force reinitialization: false + # - name: ShortwaveIRCheck + # initial value: false + # force reinitialization: false + # - name: ObsValueRangeCheck + # initial value: false + # force reinitialization: false + # - name: CloudDetection + # initial value: false + # force reinitialization: false + # - name: NSSTCheck + # initial value: false + # force reinitialization: false + # - name: GrossCheck + # initial value: false + # force reinitialization: false + # - name: InterChannelConsistency + # initial value: false + # force reinitialization: false + # - name: UseFlagCheck + # initial value: false + # force reinitialization: false + # Step 0-B: Create Derived Variables + # Assign channel wavenumbers in m-1 + - filter: Variable Assignment + assignments: + - name: MetaData/sensorCentralWavenumber + type: float + channels: *{{observation_from_jcb}}_simulated_channels + function: + name: ObsFunction/LinearCombination + options: + variables: + - name: ObsValue/radiance + channels: *{{observation_from_jcb}}_simulated_channels + coefs: [25.0] + intercept: 64475.0 + use channel numbers: true + + # Transform radiance to brightness temperature + - filter: Variable Transforms + Transform: SatBrightnessTempFromRad + transform from: + name: ObsValue/radiance + channels: *{{observation_from_jcb}}_simulated_channels + spectral variable: + name: MetaData/sensorCentralWavenumber + channels: *{{observation_from_jcb}}_simulated_channels + radiance units: wavenumber + planck1: 1.191042953e-16 + planck2: 1.4387774e-2 + + # Step 0-C: Assign Observation Error + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: assign error + error parameter vector: &{{observation_from_jcb}}_oberr [1.38, 0.81, 0.75, 0.79, 0.72, 0.74, + 0.68, 0.72, 0.65, 0.65, 0.65, 0.69, 0.64, 0.64, 0.65, 0.67, 0.62, 0.61, 0.62, + 0.64, 0.59, 0.76, 1.22, 0.78, 0.64, 0.62, 0.61, 0.69, 0.65, 0.59, 0.61, 0.59, + 0.68, 0.62, 0.68, 4.38, 3.05, 2.31, 1.56, 1.33, 1.58, 0.93, 1.67, 0.72, 0.57, + 0.58, 0.55, 0.68, 0.59, 0.68, 0.59, 0.65, 0.58, 0.62, 0.64, 0.58, 0.64, 0.55, + 0.64, 0.5, 0.82, 0.59, 0.62, 0.51, 0.64, 0.52, 0.51, 0.51, 0.76, 0.52, 0.57, + 0.55, 0.69, 0.58, 0.65, 0.61, 0.59, 0.64, 0.76, 0.72, 1.05, 0.75, 0.51, 0.65, + 1.3, 0.69, 0.93, 1.49, 1.12, 0.68, 0.66, 0.67, 0.59, 0.59, 0.69, 0.67, 0.64, + 0.62, 0.72, 0.69, 0.66, 0.79, 0.78, 0.74, 0.88, 0.77, 0.88, 0.86, 1, 0.87, + 0.85, 0.88, 0.84, 0.84, 0.84, 0.8, 0.8, 0.87, 0.98, 0.52, 0.65, 0.69, 0.61, + 0.6, 0.67, 0.79, 0.62, 0.66, 0.7, 0.65, 0.62, 0.61, 0.62, 0.53, 0.6, 0.68, + 0.95, 0.63, 0.97, 0.65, 0.98, 0.58, 0.73, 0.65, 0.85, 0.99, 0.76, 0.85, 0.97, + 0.77, 0.62, 0.63, 1.21, 1.41, 1.55, 1.78, 1.35, 1.14, 1.69, 1.79, 1.46, 1.63, + 1.94, 2.01, 1.24, 1.76, 1.26, 1.47, 1.9, 1.66, 2.13, 1.49, 1.52, 1.55, 1.96, + 2.31, 2.33, 2.32, 2.31, 2.33, 2.23, 2.33, 1.84, 2.29, 2.28, 2.28, 2.28, 2.26, + 2.26, 2.26, 2.27, 2.24, 2.23, 2.24, 2.26, 2.28, 2.28, 2.3, 2.15, 2.31, 2.37, + 2.27, 2.29, 2.29, 2.23, 2.28, 2.32, 2.32, 2.31, 2.32, 2.32, 2.31, 2.31, 2.28, + 2.29, 2.28, 2.26, 2.29, 2.27, 2.26, 2.25, 2.27, 2.24, 2.21, 2.24, 2.17, 2.18, + 2.17, 2.21, 1.99, 2.16, 2.2, 2.13, 2.12, 2.13, 2.1, 2.12, 2.11, 2.09, 2.09, + 2.08, 2.09, 2.04, 2.04, 2.1, 2.01, 2.05, 2.03, 2.06, 1.98, 1.95, 1.94, 1.91, + 1.7, 1.76, 1.77, 1.83, 2.04, 1.91, 1.99, 1.99, 2.07, 2.02, 2.04, 2.1, 2.06, + 2.18, 2.21, 2.24, 2.23, 2.23, 1.98, 2.2, 2.18, 2.18, 2.21, 2.23, 2.24, 2.24, + 2.25, 1.8, 2.24, 1.73, 1.73, 2.27, 1.67, 2.21, 1.72, 2.23, 2.23, 2.23, 2.24, + 2.23, 2.12, 2.17, 1.74, 2.02, 1.88, 1.67, 1.73, 1.83, 1.82, 1.73, 1.83, 2.19, + 1.84, 1.89, 1.6, 1.71, 1.86, 1.85, 1.84, 1.87, 1.91, 1.52, 1.95, 1.87, 1.89, + 1.91, 1.91, 1.93, 1.9, 1.91, 1.9, 1.89, 1.89, 1.91, 1.9, 1.91, 1.91, 1.91, + 1.93, 1.94, 1.91, 1.92, 1.77, 1.91, 1.95, 1.19, 1.96, 1.98, 1.94, 1.55, 1.91, + 1.92, 1.92, 1.97, 1.93, 1.99, 1.86, 1.12, 1.93, 1.92, 1.95, 1.85, 1.84, 1.91, + 1.12, 1.82, 1.82, 1.95, 1.24, 1.94, 1.96, 1.21, 1.83, 1.96, 1.36, 1.96, 1.82, + 1.92, 1.68, 1.93, 1.23, 1.96, 1.93, 1.86, 1.41, 1.16, 1.6, 1.25, 1.2, 1.65, + 1.66, 1.87, 1.94, 1.96, 1.91, 1.25, 1.93, 1.91, 1.7, 0.99, 1.81, 1.92, 1.95, + 1.5, 1.47, 1.15, 1.58, 1.18, 1.82, 1.13, 1.83, 1.91, 1.26, 1.27, 1.91, 1.45, + 1.6, 1.29, 1.94, 1.94, 1.23, 1.95, 1.21, 1.94, 1.86, 1.9, 1.33, 1.75, 2.02, + 1.98, 2.03, 1.83, 1.5, 2.04, 2.02, 1.9, 2, 2.02, 1.95, 1.93, 1.95, 1.95, 1.99, + 2, 1.94, 1.96, 1.86, 1.92, 1.88, 1.86, 1.84, 1.87, 1.77, 1.89, 1.89, 1.88, + 1.94, 1.82, 1.79, 1.86, 2.06, 2.33, 1.88, 1.86, 1.81, 1.8, 1.8, 1.86, 1.9, + 2, 2.06, 2.1, 2.2, 2, 2.16, 1.98, 1.8, 1.8, 1.85, 1.75, 2.04, 2.19, 2.14, + 2.19, 1.86, 2.1, 2.11, 2.18, 2.03, 2.28, 2.19, 2.26, 2.26, 2.21, 2.21, 2.26, + 2.33, 2.27, 2.21, 2.12, 2.23, 2.26, 2.25, 1.88, 2.26, 2.24, 2.36, 2.29, 2.35, + 2.3, 2.27, 2.08, 2.05, 2.27, 2.28, 2.27, 2.28, 1.97, 2.25, 2.25, 2.25, 2.31, + 2.28, 2.27, 2.13, 2.24, 2.28, 2.28, 2.41, 2.34, 9.32, 2.28, 2.38, 2.27, 2.27, + 2.39, 2.11, 2.09, 2.1, 2.06, 2.12, 2.08, 2, 1.93, 2.02, 2.55, 1.54, 1.64, + 1.51, 1.55, 2.82, 2.92, 2.55, 2.37, 1.85, 1.6, 1.72, 1.74, 1.79, 1.9, 1.94, + 2, 2.04, 2.08, 2.12, 2.13, 2.16, 2.18, 2.18, 2.2, 2.2, 2.41, 2.39, 2.38, 2.4, + 2.42, 2.41, 2.43, 2.45, 2.43, 2.45, 2.43, 2.4, 2.44, 2.4, 2.42, 2.43, 2.45, + 2.45, 2.45, 2.46, 2.45, 2.45, 2.43, 2.51, 2.48, 2.48, 2.53, 2.46, 2.49, 2.5, + 2.5, 2.5, 2.52, 2.52, 2.54, 2.5, 2.48, 2.5, 2.55, 2.5, 2.48, 2.5, 2.5, 2.52, + 2.52, 2.48, 2.5, 2.5, 2.52, 2.46, 2.53, 9] + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Step 1: Remove Observations from the Edge of the Scan + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: MetaData/sensorScanPosition + is_in: 5-56 + action: + name: reject + # actions: + # - name: set + # flag: ScanEdgeRemoval + # - name: reject + # Step 2: Data Thinning + - filter: Gaussian Thinning + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + horizontal_mesh: 145 + use_reduced_horizontal_grid: true + distance_norm: geodesic + priority_variable: MetaData/fractionOfClearPixelsInFOV + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + action: + name: reject + # actions: + # - name: set + # flag: Thinning + # - name: reject + # Step 3: Wavenumber Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: 7024, 7027, 7029, 7032, 7038, 7043, 7046, 7049, 7069, 7072, 7076, + 7081, 7084, 7089, 7099, 7209, 7222, 7231, 7235, 7247, 7267, 7269, 7284, 7389, + 7419, 7423, 7424, 7426, 7428, 7431, 7436, 7444, 7475, 7549, 7584, 7665, 7666, + 7831, 7836, 7853, 7865, 7885, 7888, 7912, 7950, 7972, 7980, 7995, 8007, 8015, + 8055, 8078 + where: + - variable: + name: MetaData/solarZenithAngle + maxvalue: 88.9999 + - variable: + name: water_area_fraction@GeoVaLs + minvalue: 1.0e-12 + action: + name: reject + # actions: + # - name: set + # flag: ShortwaveIRCheck + # - name: reject + # Step 4: Observation Error Inflation based on Wavenumber + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsErrorFactorWavenumIR@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + # Step 5: Observation Range Sanity Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + minvalue: 50.00001 + maxvalue: 449.99999 + # maxvalue: 100.00000 + action: + name: reject + # actions: + # - name: set + # flag: ObsValueRangeCheck + # - name: reject + # Step 6: Topography Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorTopoRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + sensor: *{{observation_from_jcb}}_sensor_id + + # Step 7: Transmittance Top Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorTransmitTopRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + # Step 8: Cloud Detection Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/CloudDetectMinResidualIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: [1, -1, -1, -1, 1, -1, -1, -1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, + 1, 1, -1, 1, 1, 1, -1, -1, 1, 1, -1, 1, -1, 1, -1, 1, -1, -1, -1, -1, -1, + -1, -1, -1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, -1, + 1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, -1, -1, + 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, 1, 1, 1, + 1, 1, 1, -1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, 1, 1, -1, -1, -1, + 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, + 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, -1, 1, 1, 1, -1, 1, 1, -1, 1, -1, + 1, -1, -1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, -1, -1, -1, 1, 1, 1, -1, + -1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, 1, -1, 1, 1, -1, 1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, + -1, -1, 1, -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, 1, + -1, 1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, -1, -1, -1, -1, -1, 1, -1, 1, -1, + 1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, + -1, -1, 1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, + -1, -1, -1, -1, -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, 1, + -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1] + use_flag_clddet: [1, -1, -1, -1, 1, -1, -1, -1, 1, -1, 1, -1, 1, -1, 1, 1, + 1, -1, 1, 1, -1, 1, 1, 1, -1, -1, 1, 1, -1, 1, -1, 1, -1, 1, -1, -1, -1, + -1, -1, -1, -1, -1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, + 1, 1, -1, 1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, + -1, -1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, 1, + 1, 1, 1, 1, 1, -1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, 1, 1, -1, + -1, -1, 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, 1, + 1, 1, 1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, -1, 1, 1, 1, -1, 1, 1, + -1, 1, -1, 1, -1, -1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, -1, -1, -1, 1, + 1, 1, -1, -1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, 1, -1, 1, 1, -1, 1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 1, -1, 1, -1, 1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, + -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1] + obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] + error parameter vector: *{{observation_from_jcb}}_oberr + maxvalue: 1.0e-12 + action: + name: reject + # actions: + # - name: set + # flag: CloudDetection + # - name: reject + # Step 9: NSST Retrieval Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: NearSSTRetCheckIR@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: [1, -1, -1, -1, 1, -1, -1, -1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, + 1, 1, -1, 1, 1, 1, -1, -1, 1, 1, -1, 1, -1, 1, -1, 1, -1, -1, -1, -1, -1, + -1, -1, -1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, -1, + 1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, -1, -1, + 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, 1, 1, 1, + 1, 1, 1, -1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, 1, 1, -1, -1, -1, + 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, + 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, -1, 1, 1, 1, -1, 1, 1, -1, 1, -1, + 1, -1, -1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, -1, -1, -1, 1, 1, 1, -1, + -1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, 1, -1, 1, 1, -1, 1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, + -1, -1, 1, -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, 1, + -1, 1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, -1, -1, -1, -1, -1, 1, -1, 1, -1, + 1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, + -1, -1, 1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, + -1, -1, -1, -1, -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, 1, + -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1] + obserr_demisf: [0.01, 0.02, 0.03, 0.02, 0.03] + obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] + maxvalue: 1.0e-12 + action: + name: reject + # actions: + # - name: set + # flag: NSSTCheck + # - name: reject + # Step 10: Surface Jacobians Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorSurfJacobianRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_demisf: [0.01, 0.02, 0.03, 0.02, 0.03] + obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] + sensor: *{{observation_from_jcb}}_sensor_id + + # Step 11: Gross check + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + function absolute threshold: + - name: ObsFunction/ObsErrorBoundIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_latitude: + name: ObsFunction/ObsErrorFactorLatRad + options: + latitude_parameters: [25.0, 0.5, 0.04, 1.0] + obserr_bound_transmittop: + name: ObsFunction/ObsErrorFactorTransmitTopRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_max: [3.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 4.0, 2.0, 2.0, 2.0, + 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 4.0, 4.0, 3.5, + 2.5, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 3.5, 2.0, 2.0, 2.0, 2.0, 2.0, + 2.0, 3.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + 2.0, 2.0, 2.0, 2.5, 2.0, 2.5, 2.5, 3.0, 2.5, 2.5, 2.5, 2.5, 3.5, 2.5, 2.5, + 3.0, 3.5, 3.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.5, 4.5, 4.5, 4.5, 4.5, 4.0, + 4.5, 4.0, 4.0, 4.5, 2.5, 3.0, 2.5, 3.0, 2.5, 3.0, 2.0, 2.5, 2.5, 3.0, 3.0, + 2.5, 3.0, 3.0, 3.0, 2.5, 2.5, 4.0, 4.5, 4.5, 5.0, 4.0, 4.0, 5.0, 5.0, 5.0, + 5.0, 5.5, 5.5, 4.0, 5.0, 4.0, 4.5, 5.5, 5.5, 6.0, 4.5, 4.5, 4.0, 5.0, 5.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 5.5, 4.5, 6.0, 5.0, 5.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 5.0, 6.0, 6.0, 6.0, 4.0, 6.0, 6.0, 6.0, 6.0, 4.5, 6.0, 6.0, 4.5, + 6.0, 6.0, 6.0, 6.0, 6.0, 5.0, 6.0, 6.0, 6.0, 5.0, 6.0, 6.0, 5.0, 6.0, 5.0, + 6.0, 6.0, 6.0, 5.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0] + error parameter vector: *{{observation_from_jcb}}_oberr + action: + name: reject + # actions: + # - name: set + # flag: GrossCheck + # - name: reject + # Step 12: Useflag Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/ChannelUseflagCheckRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: [1, -1, -1, -1, 1, -1, -1, -1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, + 1, 1, -1, 1, 1, 1, -1, -1, 1, 1, -1, 1, -1, 1, -1, 1, -1, -1, -1, -1, -1, + -1, -1, -1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, -1, + 1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, -1, -1, + 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, 1, 1, 1, + 1, 1, 1, -1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, 1, 1, -1, -1, -1, + 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, + 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, -1, 1, 1, 1, -1, 1, 1, -1, 1, -1, + 1, -1, -1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, -1, -1, -1, 1, 1, 1, -1, + -1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, 1, -1, 1, 1, -1, 1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, + -1, -1, 1, -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, 1, + -1, 1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, -1, -1, -1, -1, -1, 1, -1, 1, -1, + 1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, + -1, -1, 1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, + -1, -1, -1, -1, -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, 1, + -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1] + minvalue: 1.0e-12 + action: + name: reject + # actions: + # - name: set + # flag: UseFlagCheck + # - name: reject + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/omi_aura.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/omi_aura.yaml.j2 new file mode 100644 index 000000000..3b5cd8dd4 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/omi_aura.yaml.j2 @@ -0,0 +1,175 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: omi_aura + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [ozoneTotal] + + #obs operator: + # name: AtmVertInterpLay + # geovals: [mole_fraction_of_ozone_in_air] + # coefficients: [0.007886131] # convert from ppmv to DU + # nlevels: [1] + + # Observation Operator + # -------------------- + obs operator: + name: ColumnRetrieval + nlayers_retrieval: 1 + tracer variables: [mole_fraction_of_ozone_in_air] + isApriori: false + isAveragingKernel: false + totalNoVertice: true + stretchVertices: topbottom #options: top, bottom, topbottom, none + model units coeff: 2.241398632746E-3 + + # Observation Pre Filters (QC) + # ---------------------------- + obs pre filters: + - filter: Perform Action + filter variables: + - name: ozoneTotal + action: + name: assign error + error parameter: 6.0 + + - filter: Create Diagnostic Flags + filter variables: + - name: ozoneTotal + flags: + - name: ObsValueSanityCheck + initial value: false + force reinitialization: false + - name: RowAnomaly + initial value: false + force reinitialization: false + - name: BadScan + initial value: false + force reinitialization: false + - name: Thinning + initial value: false + force reinitialization: false + - name: RetrievalQualityCodeFlag + initial value: false + force reinitialization: false + - name: RetrievalQualityAlgorithmFlag + initial value: false + force reinitialization: false + - name: GrossCheck + initial value: false + force reinitialization: false + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # GSI read routine QC + # range sanity check + - filter: Bounds Check + filter variables: + - name: ozoneTotal + minvalue: 0 + maxvalue: 10000 + actions: + - name: set + flag: ObsValueSanityCheck + - name: reject + + # Do not use the data if row anomaly (bit 10)is 1 + - filter: RejectList + filter variables: + - name: ozoneTotal + where: + - variable: + name: MetaData/totalOzoneQualityFlag + any_bit_set_of: 9 + actions: + - name: set + flag: RowAnomaly + - name: reject + + # Scan position check: reject scan position >= 25 + - filter: RejectList + filter variables: + - name: ozoneTotal + where: + - variable: + name: MetaData/sensorScanPosition + minvalue: 25 + actions: + - name: set + flag: BadScan + - name: reject + + # Accept total_ozone_error_flag values of 0 and 1, but not any others. + - filter: RejectList + filter variables: + - name: ozoneTotal + where: + - variable: + name: MetaData/totalOzoneQualityCode + is_not_in: 0, 1 + actions: + - name: set + flag: RetrievalQualityCodeFlag + - name: reject + + # Use data with best ozone algorighm + - filter: RejectList + filter variables: + - name: ozoneTotal + where: + - variable: + name: MetaData/bestOzoneAlgorithmFlag + is_in: 3, 13 + actions: + - name: set + flag: RetrievalQualityAlgorithmFlag + - name: reject + + # Data Thinning + - filter: Gaussian Thinning + horizontal_mesh: 150 + use_reduced_horizontal_grid: true + distance_norm: geodesic + actions: + - name: set + flag: Thinning + - name: reject + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # GSI setup routine QC + # Gross check + - filter: Background Check + filter variables: + - name: ozoneTotal + threshold: 10.0 + absolute threshold: 300.0 + actions: + - name: set + flag: GrossCheck + - name: reject + + # End of Filters + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/ompsnp_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ompsnp_npp.yaml.j2 new file mode 100644 index 000000000..069771ba6 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/ompsnp_npp.yaml.j2 @@ -0,0 +1,330 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: ompsnp_npp + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsgrouping: + group variables: ["latitude"] + sort variable: "pressure" + sort order: "ascending" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [ozoneLayer] + + #obs operator: + # name: AtmVertInterpLay + # geovals: [mole_fraction_of_ozone_in_air] + # coefficients: [0.007886131] # convert from ppmv to DU + # nlevels: [22] + + # Observation Operator + # -------------------- + obs operator: + name: ColumnRetrieval + nlayers_retrieval: 1 + tracer variables: [mole_fraction_of_ozone_in_air] + isApriori: false + isAveragingKernel: false + totalNoVertice: false + stretchVertices: none #options: top, bottom, topbottom, none + # model units coeff: 2.240013904035E-3 # this number to match the gsihofx values + model units coeff: 2.241398632746E-3 # this number to match the gsihofx values (use this) + # the actual scientific conversion factor is + # 2.1415E-3 kg[O3]/m-2 to DU + # so the name of the geovals + # is also likely wrong, as it apprears to be a mass + # fraction given the conversion factor needed + + # Observation Pre Filters (QC) + # ---------------------------- + obs pre filters: + # Observation error assignment + - filter: Perform Action + filter variables: + - name: ozoneLayer + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [0.001, 10.1325, 16.00935, 25.43258, 40.32735, 63.93607, 101.325, + 160.0935, 254.3257, 403.2735, 639.3608, 1013.25, 1600.935, 2543.258, 4032.735, + 6393.607, 10132.5, 16009.35, 25432.57, 40327.35, 63936.07, 101325] + errors: [7.7236, 0.020, 0.020, 0.025, 0.080, 0.150, 0.056, 0.125, 0.200, + 0.299, 0.587, 0.864, 1.547, 2.718, 3.893, 4.353, 3.971, 4.407, 4.428, + 3.312, 2.198, 2.285] + # errors: [7.7236, 0.020, 0.020, 0.025, 0.040, 0.080, 0.156, 0.245, 0.510, 1.098, 3.917, 6.124, 6.347, 5.798, 6.843, 9.253,10.091,10.967, 8.478, 5.572, 2.638, 3.525] # operational from gfs.v16.3.9 (late 2023) + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Do not assimilation where pressure is zero + # Zero pressure indicates the data is total column ozone + - filter: RejectList + filter variables: + - name: ozoneLayer + where: + - variable: + name: MetaData/pressure + maxvalue: 0.0001 + + # Sanity check on observaton values + - filter: Bounds Check + filter variables: + - name: ozoneLayer + minvalue: 0 + maxvalue: 1000 + action: + name: reject + + # Total Ozone Quality Check (keeps 0, 2) + # 0 indentifies good data + # 2 identifies good data with a solar zenith angle > 84 degrees + - filter: RejectList + filter variables: + - name: ozoneLayer + where: + - variable: + name: MetaData/totalOzoneQuality + is_not_in: 0, 2 + + # Profile Ozone Quality Check (keeps 0, 1, 7) + # 0 : good data + # 1 : good data with a solar zenith angle > 84 degrees + # 7 : profile for which stray light correction applied + - filter: RejectList + filter variables: + - name: ozoneLayer + where: + - variable: + name: MetaData/profileOzoneQuality + is_not_in: 0, 1, 7 + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Gross error check + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 120 + action: + name: reject + where: + - variable: + name: MetaData/pressure + maxvalue: 0.001 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 30 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 30000.0 + maxvalue: 110000.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 40 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 20000.0 + maxvalue: 30000.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 44.42 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 10100.0 + maxvalue: 20000.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 57.52 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 6400.0 + maxvalue: 10100.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 69.4 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 4000.0 + maxvalue: 6400.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 70 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 2600.0 + maxvalue: 4000.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 62.73 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 1600.0 + maxvalue: 2600.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 50.52 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 1100.0 + maxvalue: 1600.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 35.9 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 700.0 + maxvalue: 1100.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 26.41 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 400.0 + maxvalue: 700.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 20.51 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 300.0 + maxvalue: 400.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 12.82 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 200.0 + maxvalue: 300.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 10 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 70.0 + maxvalue: 200.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 5 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 40.0 + maxvalue: 70.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 2 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 30.0 + maxvalue: 40.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 1 + action: + name: reject + where: + - variable: + name: MetaData/pressure + maxvalue: 30.0 + + # End of Filters + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/ompstc_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ompstc_npp.yaml.j2 new file mode 100644 index 000000000..81c64c8d7 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/ompstc_npp.yaml.j2 @@ -0,0 +1,135 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: ompstc_npp + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [ozoneTotal] + + #obs operator: + # name: AtmVertInterpLay + # geovals: [mole_fraction_of_ozone_in_air] + # coefficients: [0.007886131] # convert from ppmv to DU + # nlevels: [1] + + # Observation Operator + # -------------------- + obs operator: + name: ColumnRetrieval + nlayers_retrieval: 1 + tracer variables: [mole_fraction_of_ozone_in_air] + isApriori: false + isAveragingKernel: false + totalNoVertice: true + stretchVertices: topbottom # options: top, bottom, topbottom, none + model units coeff: 2.241398632746E-3 + + # Observation Pre Filters (QC) + # ---------------------------- + obs pre filters: + - filter: Perform Action + filter variables: + - name: ozoneTotal + action: + name: assign error + error parameter: 6.0 + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # GSI read routine QC + # range sanity check + - filter: Bounds Check + filter variables: + - name: ozoneTotal + minvalue: 0 + maxvalue: 1000 + action: + name: reject + + #- filter: Gaussian Thinning + # horizontal_mesh: 150 + # use_reduced_horizontal_grid: true + # distance_norm: geodesic + # action: + # name: reject + + # Accept total_ozone_error_flag values of 0 and 1, but not any others. + - filter: RejectList + filter variables: + - name: ozoneTotal + where: + - variable: + name: MetaData/totalOzoneQualityCode + is_not_in: 0, 1 + + - filter: RejectList + filter variables: + - name: ozoneTotal + where: + - variable: + name: MetaData/bestOzoneAlgorithmFlag + is_in: 3, 13 + + # GSI setup routine QC + - filter: RejectList + filter variables: + - name: ozoneTotal + where: + - variable: + name: MetaData/sensorScanPosition + is_in: 1, 2, 3, 4, 35 + - variable: + name: MetaData/latitude + minvalue: 50.0 + + - filter: RejectList + filter variables: + - name: ozoneTotal + where: + - variable: + name: MetaData/sensorScanPosition + is_in: 1, 2, 3, 4, 35 + - variable: + name: MetaData/latitude + maxvalue: -50.0 + + - filter: Gaussian Thinning + horizontal_mesh: 150 + use_reduced_horizontal_grid: true + distance_norm: geodesic + action: + name: reject + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + - filter: Background Check + filter variables: + - name: ozoneTotal + threshold: 10.0 + absolute threshold: 300.0 + action: + name: reject + + # End of Filters + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/satwind.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwind.yaml.j2 new file mode 100644 index 000000000..3973c3e93 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/satwind.yaml.j2 @@ -0,0 +1,982 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: satwind + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + # Observation Operator + # -------------------- + obs operator: + name: VertInterp + + # Observation Pre Filters (QC) + # ---------------------------- + obs pre filters: + # Assign the initial observation error, based on height/pressure + # Hard-wiring to prepobs_errtable.global by Type + # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML + # Type 240 (GOES SWIR): Assigned all dummy values in prepobs_errtable.global + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 0.] #Pressure (Pa) + errors: [1000000000., 1000000000.] + # Type 241 (Multi Spec. Imager LWIR): Assigned all dummy values + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 241 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 0.] #Pressure (Pa) + errors: [1000000000., 1000000000.] + # Type 242 (Himawari VIS) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 242 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, + 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., + 7., 7., 7.] + # Type 243 (MVIRI/SEVIRI VIS) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 243 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, + 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., + 7., 7., 7.] + # Type 244 (AVHRR LWIR) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 244 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, + 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., + 7., 7., 7.] + # Type 245 (GOES LWIR): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 245 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, + 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., + 14., 14., 14., 14., 14., 14.] + # Type 246 (GOES cloud-top WV): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 246 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, + 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., + 14., 14., 14., 14., 14., 14.] + # Type 247 (GOES clear-sky WV): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 247 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, + 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., + 14., 14., 14., 14., 14., 14.] + # Type 248 (GOES Sounder cloud-top WV): Assigned all dummy values + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 248 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 0.] #Pressure (Pa) + errors: [1000000000., 1000000000.] + # Type 249 (GOES Sounder clear-sky WV): Assigned all dummy values + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 249 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 0.] #Pressure (Pa) + errors: [1000000000., 1000000000.] + # Type 250 (Himawari AHI WV, cloud-top or clear-sky) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 250 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, + 5., 7., 7.3, 7.6, 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., + 8., 8., 8.] + # Type 251 (GOES VIS): Assigned all dummy values + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 251 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 0.] #Pressure (Pa) + errors: [1000000000., 1000000000.] + # Type 252 (Himawari AHI LWIR) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 252 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, + 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., + 7., 7., 7.] + # Type 253 (MVIRI/SEVERI LWIR) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 253 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, + 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., + 7., 7., 7.] + # Type 254 (MVIRI/SEVIRI WV, both cloud-top and clear-sky) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 254 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4.5, 6.1, + 6., 6.5, 7.3, 7.6, 7., 7.5, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., + 7., 7., 7.] + # Type 255 (GOES low-level picture triplet cloud drift): According to prepbufr table this should no longer exist?? + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 255 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, + 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., + 7., 7., 7.] + # Type 256 (Multi Spec. Imager WV, both clear-sky and cloud-top): Assigned all dummy values + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 256 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 0.] #Pressure (Pa) + errors: [1000000000., 1000000000.] + # Type 257 (MODIS LWIR) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 257 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, + 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., + 7., 7., 7.] + # Type 258 (MODIS cloud-top WV): Some levels assigned dummy values + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 258 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [1000000000., 1000000000., 4., 4.1, 5., 6., 6.3, 6.6, 7., 7., 7., + 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7.] + # Type 259 (MODIS clear-sky WV): Some levels assigned dummy values + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 259 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [1000000000., 1000000000., 4., 4.1, 5., 6., 6.3, 6.6, 7., 7., 7., + 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7.] + # Type 260 (VIIRS LWIR): All levels assigned dummy values in prepobs_errtable.global, HOWEVER the GSI values appear + # to be a standard profile (borrowed from e.g., Type=244). Using the standard profile here. + # It"s possibly that my prepobs_errtable.global file is out-of-date. + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 260 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, + 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., + 7., 7., 7.] + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # + # sanity-check criteria + # + # Observation Range Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130. + maxvalue: 130. + action: + name: reject + # Velocity Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130. + action: + name: reject + # + # preQC (read_satwnd) criteria + # + # EUMETSAT winds: satelliteIdentifer [50–79] (>49, <80) + # Reject obs with satelliteZenithAngle > 68 deg + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + minvalue: 49. + maxvalue: 80. + test variables: + - name: MetaData/satelliteZenithAngle + maxvalue: 68. + action: + name: reject + # Reject obs with windComputationMethod = 5 (clear-sky WV AMV) + # CLEARED + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + minvalue: 49. + maxvalue: 80. + test variables: + - name: MetaData/windComputationMethod + maxvalue: 4. + action: + name: reject + # Reject obs with qualityInformationWithoutForecast < 85. + # CLEARED + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + minvalue: 49. + maxvalue: 80. + test variables: + - name: MetaData/qualityInformationWithoutForecast + minvalue: 85. + action: + name: reject + # JMA: satelliteIdentifier [100–199] (>99, <200) + # Reject obs with satelliteZenithAngle > 68 deg + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + minvalue: 99. + maxvalue: 200. + test variables: + - name: MetaData/satelliteZenithAngle + maxvalue: 68. + action: + name: reject + # Reject obs with windComputationMethod = 5 (clear-sky WV AMV) + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + minvalue: 99. + maxvalue: 200. + test variables: + - name: MetaData/windComputationMethod + maxvalue: 4. + action: + name: reject + # Reject obs with qualityInformationWithoutForecast < 85. + # CLEARED + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + minvalue: 99. + maxvalue: 200. + test variables: + - name: MetaData/qualityInformationWithoutForecast + minvalue: 85. + action: + name: reject + # NESDIS: satelliteIdentifier [250–299] (>249, <300) + # Reject obs with satelliteZenithAngle > 68 deg + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + minvalue: 249. + maxvalue: 300. + test variables: + - name: MetaData/satelliteZenithAngle + maxvalue: 68. + action: + name: reject + # Reject obs with qualityInformationWithoutForecast < 90. OR > 100. + # CLEARED + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + minvalue: 249. + maxvalue: 300. + test variables: + - name: MetaData/qualityInformationWithoutForecast + minvalue: 90. + maxvalue: 100. + action: + name: reject + # Reject obs with pressure < 15000. + # CLEARED + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + minvalue: 249. + maxvalue: 300. + test variables: + - name: MetaData/pressure + minvalue: 15000. + action: + name: reject + # Reject obs with pressure < 70000. when Type=251 + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + minvalue: 249. + maxvalue: 300. + - variable: ObsType/windEastward + is_in: 251 + test variables: + - name: MetaData/pressure + minvalue: 70000. + action: + name: reject + # Reject obs with pressure > 30000. when Type=246 + # CLEARED + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + minvalue: 249. + maxvalue: 300. + - variable: ObsType/windEastward + is_in: 246 + test variables: + - name: MetaData/pressure + maxvalue: 30000. + action: + name: reject + # Reject obs with pressure > 85000. when isli=1 (land surface) + # CLEARED + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + minvalue: 249. + maxvalue: 300. + - variable: GeoVaLs/land_type_index_NPOESS + minvalue: 1. + maxvalue: 1. + test variables: + - name: MetaData/pressure + maxvalue: 85000. + action: + name: reject + # Reject obs with pct1 (Coeff. of Var.) outside of 0.04–0.5, Type [240,245,246,251] ONLY + # CLEARED + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + minvalue: 249. + maxvalue: 300. + - variable: ObsType/windEastward + is_in: 240,245,246,251 + test variables: + - name: MetaData/coefficientOfVariation + minvalue: 0.04 + maxvalue: 0.5 + action: + name: reject + # NESDIS obs are also subject to the experr_norm test defined as: + # + # if (10. - 0.1*(expectedError))/(ob_speed)>0.9, or ob_speed<0.1, reject, applies to Types [240,245,246,247,251] + # + # This is not implemented in the YAML file because we do not have the capability to compute + # the norm, lacking the required math operators. Instead, this will likely have to be + # implemented as an ObsFunction like the SatWindsLNVDCheck test. + # + # setupw criteria + # + # Reject any ob Type [240–260] when pressure greater than 950 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240-260 + test variables: + - name: MetaData/pressure + maxvalue: 95001. + action: + name: reject + # GOES IR (245) reject when pressure between 399 and 801 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/pressure + minvalue: 39901. + maxvalue: 80099. + - variable: ObsType/windEastward + is_in: 245 + action: + name: reject + # JMA IR (252) reject when pressure between 499 and 801 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/pressure + minvalue: 49901. + maxvalue: 80099. + - variable: ObsType/windEastward + is_in: 252 + action: + name: reject + # EUMETSAT IR (253) reject when pressure between 401 and 801 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/pressure + minvalue: 40101. + maxvalue: 80099. + - variable: ObsType/windEastward + is_in: 253 + action: + name: reject + # GOES WV (246, 250, 254), reject when pressure greater than 399 mb. + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 246, 250, 254 + test variables: + - name: MetaData/pressure + maxvalue: 39900. + action: + name: reject + # EUMET (242) and JMA (243) vis, reject when pressure less than 700 mb. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 242, 243 + test variables: + - name: MetaData/pressure + minvalue: 70000. + action: + name: reject + # MODIS-Aqua/Terra (257) and (259), reject when pressure less than 249 mb. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 257,259 + test variables: + - name: MetaData/pressure + minvalue: 24900. + action: + name: reject + # MODIS-Aqua/Terra (258) and (259), reject when pressure greater than 600 mb. + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + # maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 258, 259 + test variables: + - name: MetaData/pressure + maxvalue: 60000. + action: + name: reject + # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + reference: GeoVaLs/tropopause_pressure + value: MetaData/pressure + minvalue: -5000. # 50 hPa above tropopause level, negative p-diff + action: + name: reject + # GOES (247) reject any observation with a /=0 surface type (non-water + # surface) within 110 hPa of the surface pressure (as part of the LNVD + # check). + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: + name: GeoVaLs/land_type_index_NPOESS + minvalue: 1. + - variable: + name: ObsType/windEastward + is_in: 247 + reference: GeoVaLs/surface_pressure + value: MetaData/pressure + maxvalue: -11000. # within 110 hPa above surface pressure, negative p-diff + action: + name: reject + # AVHRR (244), MODIS (257,258,259), and VIIRS (260) reject any + # observation with a /=0 surface type (non-water surface) within + # 200 hPa of the surface pressure (as part of the LNVD check). + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: + name: GeoVaLs/land_type_index_NPOESS + minvalue: 1. + - variable: + name: ObsType/windEastward + is_in: 244, 257-260 + reference: GeoVaLs/surface_pressure + value: MetaData/pressure + maxvalue: -20000. # within 200 hPa above surface pressure, negative p-diff + action: + name: reject + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Reject GOES (247) when difference of wind direction is more than 50 degrees. + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 247 + test variables: + - name: ObsFunction/WindDirAngleDiff + maxvalue: 50. + action: + name: reject + # AVHRR (244), MODIS (257,258,259), VIIRS (260), GOES (247) use a LNVD check. + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 244, 247, 257-260 + test variables: + - name: ObsFunction/SatWindsLNVDCheck + maxvalue: 3. + action: + name: reject + # + # All satwinds subject to a SPDB check (function may be broken?) + # We are ignoring this filter for now, and will come back to + # the issue of how to handle this SPDB check later. For now, + # observations that are GSI-rejected based on this test are + # not being considered when checking acceptance compliance. + #- filter: Bounds Check + # filter variables: + # - name: windEastward + # - name: windNorthward + # test variables: + # - name: ObsFunction/SatWindsSPDBCheck + # options: + # error_min: 1.4 + # error_max: 20.0 + # maxvalue: 1.75 + # action: + # name: reject + # defer to post: true + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: Identity + + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/satwind_abi_goes-16.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwind_abi_goes-16.yaml.j2 new file mode 100644 index 000000000..e7b9c62e5 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/satwind_abi_goes-16.yaml.j2 @@ -0,0 +1,601 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: satwind_goes-16 + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + # Observation Operator + # -------------------- + obs operator: + name: VertInterp + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: VertInterp + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: false + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Assign the initial observation error, based on height/pressure + # Hard-wiring to prepobs_errtable.global by Type + # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML + # Type 240 (GOES SWIR): Assigned all dummy values in prepobs_errtable.global + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 0.] #Pressure (Pa) + errors: [1000000000., 1000000000.] + # Type 245 (GOES LWIR): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 245 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, + 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., + 14., 14., 14., 14., 14., 14.] + # Type 246 (GOES cloud-top WV): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 246 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, + 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., + 14., 14., 14., 14., 14., 14.] + # Type 247 (GOES clear-sky WV): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 247 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, + 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., + 14., 14., 14., 14., 14., 14.] + # Type 251 (GOES VIS): Assigned all dummy values + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 251 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 0.] #Pressure (Pa) + errors: [1000000000., 1000000000.] + # sanity-check criteria + # Observation Range Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130. + maxvalue: 130. + action: + name: reject + + # Velocity Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130. + action: + name: reject + + # GSI read routine QC (part-1) + # Exclude Type 251 (VIS) --- obs tossed without passing to setup routine + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 251 + action: + name: reject + + # Exclude data with satellite zenith angle > 68 for all types --- obs tossed without passing to setup routine + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: MetaData/satelliteZenithAngle + maxvalue: 68. + action: + name: reject + + # Reject obs with pressure < 12500 pa --- obs tosed without passing to setup routine + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + test variables: + - name: MetaData/pressure + minvalue: 12500. + action: + name: reject + + # Exclude data over non-water surface type where latitude > 20N for Type 240 (IRSW) and Type 245 (IRLW) --- obs tossed and not passed to setup routine + # Notes: This check was missing, so added (eliu) + # Replace land_type_index_NPOSS with water_area_fraction (eliu) + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240, 245 + - variable: MetaData/latitude + minvalue: 20. + test variables: + - name: GeoVaLs/water_area_fraction + minvalue: 0.99 + action: + name: reject + + # GSI read routine QC (part-2) + # Reject obs with qualityInformationWithoutForecast < 90. OR > 100. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + test variables: + - name: MetaData/qualityInformationWithoutForecast + minvalue: 90. + maxvalue: 100. + action: + name: reject + + # Reject obs with pressure < 15000 pa. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + test variables: + - name: MetaData/pressure + minvalue: 15000. + action: + name: reject + + # Reject obs with pressure < 70000 pa. when Type=251 + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + - variable: ObsType/windEastward + is_in: 251 + test variables: + - name: MetaData/pressure + minvalue: 70000. + action: + name: reject + + # Reject obs with pressure > 30000. when Type=246 + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + - variable: ObsType/windEastward + is_in: 246 + test variables: + - name: MetaData/pressure + maxvalue: 30000. + action: + name: reject + + # Reject obs with pressure > 85000. when isli=1 (land surface) + # Notes: Replace land_type_index_NPOESS with land_area_fraction (eliu) + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + - variable: GeoVaLs/land_area_fraction + minvalue: 0.99 + test variables: + - name: MetaData/pressure + maxvalue: 85000. + action: + name: reject + + # Reject obs with pct1 (Coeff. of Var.) outside of 0.04–0.5, Type [240,245,246,251] ONLY + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + - variable: ObsType/windEastward + is_in: 240, 245, 246, 251 + test variables: + - name: MetaData/coefficientOfVariation + minvalue: 0.04 + maxvalue: 0.5 + action: + name: reject + + # NESDIS obs are also subject to the experr_norm test defined as: + # + # if (10. - 0.1*(expectedError))/(ob_speed)>0.9, or ob_speed<0.1, reject, applies to NESDIS winds + # + # CLEARED: With caveat that float precision/handling differences can generate different acceptance criteria + # between UFO and GSI for observations with an experr_norm value right around the maxvalue. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + test variables: + - name: ObsFunction/SatWindsErrnormCheck + maxvalue: 0.9 + action: + name: reject + + # Reject all Type=240 (GOES SWIR) AMVs: These are not currently assimilated in GSI and they have missing-values + # assigned to ob-errors + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240 + action: + name: reject + + # GSI setupw routine QC + # Reject any ob Type [240–260] when pressure greater than 950 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240-260 + test variables: + - name: MetaData/pressure + maxvalue: 95001. + action: + name: reject + + # GOES IR (245) reject when pressure between 399 and 801 mb. + # # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/pressure + minvalue: 39901. + maxvalue: 80099. + - variable: ObsType/windEastward + is_in: 245 + action: + name: reject + + # GOES WV (246, 250, 254), reject when pressure greater than 399 mb. + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 246, 250, 254 + test variables: + - name: MetaData/pressure + maxvalue: 39900. + action: + name: reject + + # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation + # Turn this check off for now. + # Need to check if troposphere pressure was implemented correctly in fv3-jed + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + reference: GeoVaLs/tropopause_pressure + value: MetaData/pressure + minvalue: -5000. # 50 hPa above tropopause level, negative p-diff + action: + name: reject + + # GOES (247) reject any observation with a /=0 surface type (non-water + # surface) within 110 hPa of the surface pressure (as part of the LNVD # check). + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + # Notes (eliu): Replace land_type_index_NPOESS with land_area_fraction. + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + - variable: + name: ObsType/windEastward + is_in: 247 + reference: GeoVaLs/surface_pressure + value: MetaData/pressure + maxvalue: -11000. # within 110 hPa above surface pressure, negative p-diff + action: + name: reject + + # Reject GOES (247) when difference of wind direction is more than 50 degrees. + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 247 + test variables: + - name: ObsFunction/WindDirAngleDiff + maxvalue: 50. + action: + name: reject + + # All satwinds must adjust errors based on ObsErrorFactorPressureCheck + # prior to the SPDB check (i.e. the gross-error check). The gross-error + # check uses the adjusted errors for error-bound tightening and rejection, + # so this check has to come first. This check will inflate errors for obs + # that are too close to either the model top or bottom. + # Notes (eliu): GMAO added a required parameter: adjusted_error_name. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + surface_obs: false + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All satwinds subject to a gross-error check that contains significant + # modifiers for satwinds with a negative speed-bias. ALL wind gross-error + # checks are currently being done by the SatWindsSPDBCheck. + # CLEARED + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, + 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, + 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, + 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, + 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, + 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, + 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, + 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, + 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don"t want to inflate errors for duplication if one + # of the duplicates should be rejected. + # Notes (eliu): ObsErrorFactorDuplicateCheck obsfunction requires PreUseFlag (usage parameter from read_satwnd.f90). + # : Turn off duplicate check for now. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + # We are extending this to an additional filter that inflates final ob-errors across-the-board by + # 1/0.8 = 1.25. This is caused by the GSI value of nvqc being set to .true. in the global operational + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this nvqc functionality were to be switched off (i.e. if variational qc were to be turned off), + # you would want to remove this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + # End of Filters + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/satwind_abi_goes-17.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwind_abi_goes-17.yaml.j2 new file mode 100644 index 000000000..3dd8a44f2 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/satwind_abi_goes-17.yaml.j2 @@ -0,0 +1,601 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: satwind_goes-17 + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + # Observation Operator + # -------------------- + obs operator: + name: VertInterp + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: VertInterp + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: false + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Assign the initial observation error, based on height/pressure + # Hard-wiring to prepobs_errtable.global by Type + # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML + # Type 240 (GOES SWIR): Assigned all dummy values in prepobs_errtable.global + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 0.] #Pressure (Pa) + errors: [1000000000., 1000000000.] + # Type 245 (GOES LWIR): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 245 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, + 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., + 14., 14., 14., 14., 14., 14.] + # Type 246 (GOES cloud-top WV): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 246 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, + 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., + 14., 14., 14., 14., 14., 14.] + # Type 247 (GOES clear-sky WV): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 247 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, + 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., + 14., 14., 14., 14., 14., 14.] + # Type 251 (GOES VIS): Assigned all dummy values + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 251 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 0.] #Pressure (Pa) + errors: [1000000000., 1000000000.] + # sanity-check criteria + # Observation Range Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130. + maxvalue: 130. + action: + name: reject + + # Velocity Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130. + action: + name: reject + + # GSI read routine QC (part-1) + # Exclude Type 251 (VIS) --- obs tossed without passing to setup routine + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 251 + action: + name: reject + + # Exclude data with satellite zenith angle > 68 for all types --- obs tossed without passing to setup routine + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: MetaData/satelliteZenithAngle + maxvalue: 68. + action: + name: reject + + # Reject obs with pressure < 12500 pa --- obs tosed without passing to setup routine + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + test variables: + - name: MetaData/pressure + minvalue: 12500. + action: + name: reject + + # Exclude data over non-water surface type where latitude > 20N for Type 240 (IRSW) and Type 245 (IRLW) --- obs tossed and not passed to setup routine + # Notes: This check was missing, so added (eliu) + # Replace land_type_index_NPOSS with water_area_fraction (eliu) + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240, 245 + - variable: MetaData/latitude + minvalue: 20. + test variables: + - name: GeoVaLs/water_area_fraction + minvalue: 0.99 + action: + name: reject + + # GSI read routine QC (part-2) + # Reject obs with qualityInformationWithoutForecast < 90. OR > 100. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + test variables: + - name: MetaData/qualityInformationWithoutForecast + minvalue: 90. + maxvalue: 100. + action: + name: reject + + # Reject obs with pressure < 15000 pa. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + test variables: + - name: MetaData/pressure + minvalue: 15000. + action: + name: reject + + # Reject obs with pressure < 70000 pa. when Type=251 + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + - variable: ObsType/windEastward + is_in: 251 + test variables: + - name: MetaData/pressure + minvalue: 70000. + action: + name: reject + + # Reject obs with pressure > 30000. when Type=246 + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + - variable: ObsType/windEastward + is_in: 246 + test variables: + - name: MetaData/pressure + maxvalue: 30000. + action: + name: reject + + # Reject obs with pressure > 85000. when isli=1 (land surface) + # Notes: Replace land_type_index_NPOESS with land_area_fraction (eliu) + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + - variable: GeoVaLs/land_area_fraction + minvalue: 0.99 + test variables: + - name: MetaData/pressure + maxvalue: 85000. + action: + name: reject + + # Reject obs with pct1 (Coeff. of Var.) outside of 0.04–0.5, Type [240,245,246,251] ONLY + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + - variable: ObsType/windEastward + is_in: 240, 245, 246, 251 + test variables: + - name: MetaData/coefficientOfVariation + minvalue: 0.04 + maxvalue: 0.5 + action: + name: reject + + # NESDIS obs are also subject to the experr_norm test defined as: + # + # if (10. - 0.1*(expectedError))/(ob_speed)>0.9, or ob_speed<0.1, reject, applies to NESDIS winds + # + # CLEARED: With caveat that float precision/handling differences can generate different acceptance criteria + # between UFO and GSI for observations with an experr_norm value right around the maxvalue. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + test variables: + - name: ObsFunction/SatWindsErrnormCheck + maxvalue: 0.9 + action: + name: reject + + # Reject all Type=240 (GOES SWIR) AMVs: These are not currently assimilated in GSI and they have missing-values + # assigned to ob-errors + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240 + action: + name: reject + + # GSI setupw routine QC + # Reject any ob Type [240–260] when pressure greater than 950 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240-260 + test variables: + - name: MetaData/pressure + maxvalue: 95001. + action: + name: reject + + # GOES IR (245) reject when pressure between 399 and 801 mb. + # # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/pressure + minvalue: 39901. + maxvalue: 80099. + - variable: ObsType/windEastward + is_in: 245 + action: + name: reject + + # GOES WV (246, 250, 254), reject when pressure greater than 399 mb. + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 246, 250, 254 + test variables: + - name: MetaData/pressure + maxvalue: 39900. + action: + name: reject + + # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation + # Turn this check off for now. + # Need to check if troposphere pressure was implemented correctly in fv3-jed + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + reference: GeoVaLs/tropopause_pressure + value: MetaData/pressure + minvalue: -5000. # 50 hPa above tropopause level, negative p-diff + action: + name: reject + + # GOES (247) reject any observation with a /=0 surface type (non-water + # surface) within 110 hPa of the surface pressure (as part of the LNVD # check). + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + # Notes (eliu): Replace land_type_index_NPOESS with land_area_fraction. + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + - variable: + name: ObsType/windEastward + is_in: 247 + reference: GeoVaLs/surface_pressure + value: MetaData/pressure + maxvalue: -11000. # within 110 hPa above surface pressure, negative p-diff + action: + name: reject + + # Reject GOES (247) when difference of wind direction is more than 50 degrees. + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 247 + test variables: + - name: ObsFunction/WindDirAngleDiff + maxvalue: 50. + action: + name: reject + + # All satwinds must adjust errors based on ObsErrorFactorPressureCheck + # prior to the SPDB check (i.e. the gross-error check). The gross-error + # check uses the adjusted errors for error-bound tightening and rejection, + # so this check has to come first. This check will inflate errors for obs + # that are too close to either the model top or bottom. + # Notes (eliu): GMAO added a required parameter: adjusted_error_name. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + surface_obs: false + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All satwinds subject to a gross-error check that contains significant + # modifiers for satwinds with a negative speed-bias. ALL wind gross-error + # checks are currently being done by the SatWindsSPDBCheck. + # CLEARED + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, + 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, + 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, + 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, + 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, + 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, + 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, + 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, + 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don"t want to inflate errors for duplication if one + # of the duplicates should be rejected. + # Notes (eliu): ObsErrorFactorDuplicateCheck obsfunction requires PreUseFlag (usage parameter from read_satwnd.f90). + # : Turn off duplicate check for now. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + # We are extending this to an additional filter that inflates final ob-errors across-the-board by + # 1/0.8 = 1.25. This is caused by the GSI value of nvqc being set to .true. in the global operational + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this nvqc functionality were to be switched off (i.e. if variational qc were to be turned off), + # you would want to remove this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + # End of Filters + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/satwind_ahi_h8.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwind_ahi_h8.yaml.j2 new file mode 100644 index 000000000..500f09148 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/satwind_ahi_h8.yaml.j2 @@ -0,0 +1,432 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: satwind_ahi_h8 + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + # Observation Operator + # -------------------- + obs operator: + name: VertInterp + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: VertInterp + + # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI"s thinning of AHI/Himawari-8 satwinds + # results in more JEDI satwinds in the diag file than in GSI, but far fewer JEDI satwinds assimilated than + # GSI. JEDI under-counts assimilated winds by roughly 25-40%, relative to GSI, and this under-count is not + # even including the temporal thinning which is applied in GSI but not JEDI (by this filter below). See + # GDASApp Issue #741 for details: https://github.com/NOAA-EMC/GDASApp/issues/741 + #obs pre filters: + #- filter: Gaussian Thinning + # horizontal_mesh: 200 + # vertical_mesh: 10000 + # use_reduced_horizontal_grid: true + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: false + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Assign the initial observation error, based on height/pressure + # Hard-wiring to prepobs_errtable.global by Type + # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML + # Type 242 (Himawari VIS) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 242 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, + 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., + 7., 7., 7.] + # Type 250 (Himawari AHI WV, cloud-top or clear-sky) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 250 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, + 5., 7., 7.3, 7.6, 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., + 8., 8., 8.] + # Type Type 252 (Himawari AHI LWIR) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 252 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, + 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., + 7., 7., 7.] + # sanity-check criteria + # Observation Range Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130. + maxvalue: 130. + action: + name: reject + + # Velocity Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130. + action: + name: reject + + # GSI read routine QC (part-1) + # Exclude Type 250 with windComputationMethod==5 (clear-sky WV) --- obs tossed without passing to setup routine + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windNorthward + is_in: 250 + - variable: MetaData/windComputationMethod + is_in: 5 + action: + name: reject + + # Exclude data with satellite zenith angle > 68 for all types --- obs tossed without passing to setup routine + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: MetaData/satelliteZenithAngle + maxvalue: 68. + action: + name: reject + + # GSI read routine QC (part-2) + # Reject obs with qualityInformationWithoutForecast < 85. (also > 100., which is missing data) + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: MetaData/qualityInformationWithoutForecast + minvalue: 85. + maxvalue: 100. + action: + name: reject + + # Reject Type 252 (IR) winds with a /=0 surface type (non-water surface) when latitude > 20. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + - variable: + name: ObsType/windEastward + is_in: 252 + test variables: + - name: MetaData/latitude + maxvalue: 20. + action: + name: reject + + # GSI setupw routine QC + # Reject any ob Type [240–260] when pressure greater than 950 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240-260 + test variables: + - name: MetaData/pressure + maxvalue: 95001. + action: + name: reject + + # IR (Type 242), reject when pressure is less than 700 mb + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 242 + test variables: + - name: MetaData/pressure + minvalue: 70000. + action: + name: reject + + # cloud-top WV (Type 250), reject when pressure greater than 399 mb. + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 250 + test variables: + - name: MetaData/pressure + maxvalue: 39900. + action: + name: reject + + # JMA IR (252) reject when pressure between 499 and 801 mb. + # PERFORM ACTION: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/pressure + minvalue: 49901. + maxvalue: 80099. + - variable: ObsType/windEastward + is_in: 252 + action: + name: reject + + # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation + # Turn this check off for now. + # Need to check if troposphere pressure was implemented correctly in fv3-jed + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + reference: GeoVaLs/tropopause_pressure + value: MetaData/pressure + minvalue: -5000. # 50 hPa above tropopause level, negative p-diff + action: + name: reject + + # All satwinds must adjust errors based on ObsErrorFactorPressureCheck + # prior to the SPDB check (i.e. the gross-error check). The gross-error + # check uses the adjusted errors for error-bound tightening and rejection, + # so this check has to come first. This check will inflate errors for obs + # that are too close to either the model top or bottom. + # Notes (eliu): GMAO added a required parameter: adjusted_error_name. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + surface_obs: false + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All satwinds subject to a gross-error check that contains significant + # modifiers for satwinds with a negative speed-bias. ALL wind gross-error + # checks are currently being done by the SatWindsSPDBCheck. + # CLEARED + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, + 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, + 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, + 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, + 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, + 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, + 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, + 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, + 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don"t want to inflate errors for duplication if one + # of the duplicates should be rejected. + # Notes (eliu): ObsErrorFactorDuplicateCheck obsfunction requires PreUseFlag (usage parameter from read_satwnd.f90). + # : Turn off duplicate check for now. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + # We are extending this to an additional filter that inflates final ob-errors across-the-board by + # 1/0.8 = 1.25. This is caused by the GSI value of nvqc being set to .true. in the global operational + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this nvqc functionality were to be switched off (i.e. if variational qc were to be turned off), + # you would want to remove this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + # End of Filters + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/satwind_seviri_m11.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwind_seviri_m11.yaml.j2 new file mode 100644 index 000000000..884562104 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/satwind_seviri_m11.yaml.j2 @@ -0,0 +1,439 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: satwind_seviri_m11 + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + # Observation Operator + # -------------------- + obs operator: + name: VertInterp + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: VertInterp + + # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI"s thinning of SEVIRI/METEOSAT-8 satwinds + # results in more JEDI satwinds in the diag file than in GSI, but far fewer JEDI satwinds assimilated than + # GSI. JEDI under-counts assimilated winds by roughly 25-40%, relative to GSI, and this under-count is not + # even including the temporal thinning which is applied in GSI but not JEDI (by this filter below). See + # GDASApp Issue #758 for details: https://github.com/NOAA-EMC/GDASApp/issues/758 + #obs pre filters: + #- filter: Gaussian Thinning + # where: + # - variable: ObsType/windEastward + # is_in: 243, 253 + # horizontal_mesh: 200 + # vertical_mesh: 10000 + # use_reduced_horizontal_grid: true + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: false + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Assign the initial observation error, based on height/pressure + # Hard-wiring to prepobs_errtable.global by Type + # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML + # Type 243 (MVIRI/SEVIRI VIS) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 243 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, + 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., + 7., 7., 7.] + # Type 253 (MVIRI/SEVERI LWIR) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 253 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, + 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., + 7., 7., 7.] + # Type 254 (MVIRI/SEVIRI WV, both cloud-top and clear-sky) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 254 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4.5, 6.1, + 6., 6.5, 7.3, 7.6, 7., 7.5, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., + 7., 7., 7.] + # sanity-check criteria + # Observation Range Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130. + maxvalue: 130. + action: + name: reject + + # Velocity Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130. + action: + name: reject + + # GSI read routine QC (part-1) + # Exclude Type 254 with windComputationMethod==5 (clear-sky WV) --- obs tossed without passing to setup routine + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windNorthward + is_in: 254 + - variable: MetaData/windComputationMethod + is_in: 5 + action: + name: reject + + # Exclude data with satellite zenith angle > 68 for all types --- obs tossed without passing to setup routine + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: MetaData/satelliteZenithAngle + maxvalue: 68. + action: + name: reject + + # Exclude data over non-water surface type where latitude > 20N for Type 253 (IRLW) --- obs tossed and not passed to setup routine + # Notes: This check was missing, so added (eliu) + # Replace land_type_index_NPOSS with water_area_fraction (eliu) + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 253 + - variable: MetaData/latitude + minvalue: 20. + test variables: + - name: GeoVaLs/water_area_fraction + minvalue: 0.99 + action: + name: reject + + # GSI read routine QC (part-2) + # Reject obs with qualityInformationWithoutForecast < 85. OR > 100. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: MetaData/qualityInformationWithoutForecast + minvalue: 85. + maxvalue: 100. + action: + name: reject + + # GSI setupw routine QC + # Reject any ob Type [240–260] when pressure greater than 950 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240-260 + test variables: + - name: MetaData/pressure + maxvalue: 95001. + action: + name: reject + + # EUMETSAT IR (253) reject when pressure between 401 and 801 mb. + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/pressure + minvalue: 40101. + maxvalue: 80099. + - variable: ObsType/windEastward + is_in: 253 + action: + name: reject + + # EUMET VIS (243) reject when pressure less than 700 mb. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 243 + test variables: + - name: MetaData/pressure + minvalue: 70000. + action: + name: reject + + # EUMET WV (254) reject when pressure greater than 399 mb. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 254 + test variables: + - name: MetaData/pressure + maxvalue: 39900. + action: + name: reject + + # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation + # Turn this check off for now. + # Need to check if troposphere pressure was implemented correctly in fv3-jed + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + reference: GeoVaLs/tropopause_pressure + value: MetaData/pressure + minvalue: -5000. # 50 hPa above tropopause level, negative p-diff + action: + name: reject + + # All satwinds must adjust errors based on ObsErrorFactorPressureCheck + # prior to the SPDB check (i.e. the gross-error check). The gross-error + # check uses the adjusted errors for error-bound tightening and rejection, + # so this check has to come first. This check will inflate errors for obs + # that are too close to either the model top or bottom. + # Notes (eliu): GMAO added a required parameter: adjusted_error_name. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + surface_obs: false + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All satwinds subject to a gross-error check that contains significant + # modifiers for satwinds with a negative speed-bias. ALL wind gross-error + # checks are currently being done by the SatWindsSPDBCheck. + # CLEARED + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, + 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, + 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, + 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, + 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, + 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, + 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, + 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, + 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don"t want to inflate errors for duplication if one + # of the duplicates should be rejected. + # Notes (eliu): ObsErrorFactorDuplicateCheck obsfunction requires PreUseFlag (usage parameter from read_satwnd.f90). + # : Turn off duplicate check for now. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + # We are extending this to an additional filter that inflates final ob-errors across-the-board by + # 1/0.8 = 1.25. This is caused by the GSI value of nvqc being set to .true. in the global operational + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this nvqc functionality were to be switched off (i.e. if variational qc were to be turned off), + # you would want to remove this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + # End of Filters + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/satwind_seviri_m8.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwind_seviri_m8.yaml.j2 new file mode 100644 index 000000000..86c0d4259 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/satwind_seviri_m8.yaml.j2 @@ -0,0 +1,439 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: satwind_seviri_m8 + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + # Observation Operator + # -------------------- + obs operator: + name: VertInterp + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: VertInterp + + # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI"s thinning of SEVIRI/METEOSAT-8 satwinds + # results in more JEDI satwinds in the diag file than in GSI, but far fewer JEDI satwinds assimilated than + # GSI. JEDI under-counts assimilated winds by roughly 25-40%, relative to GSI, and this under-count is not + # even including the temporal thinning which is applied in GSI but not JEDI (by this filter below). See + # GDASApp Issue #758 for details: https://github.com/NOAA-EMC/GDASApp/issues/758 + #obs pre filters: + #- filter: Gaussian Thinning + # where: + # - variable: ObsType/windEastward + # is_in: 243, 253 + # horizontal_mesh: 200 + # vertical_mesh: 10000 + # use_reduced_horizontal_grid: true + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: false + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Assign the initial observation error, based on height/pressure + # Hard-wiring to prepobs_errtable.global by Type + # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML + # Type 243 (MVIRI/SEVIRI VIS) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 243 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, + 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., + 7., 7., 7.] + # Type 253 (MVIRI/SEVERI LWIR) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 253 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, + 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., + 7., 7., 7.] + # Type 254 (MVIRI/SEVIRI WV, both cloud-top and clear-sky) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 254 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4.5, 6.1, + 6., 6.5, 7.3, 7.6, 7., 7.5, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., + 7., 7., 7.] + # sanity-check criteria + # Observation Range Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130. + maxvalue: 130. + action: + name: reject + + # Velocity Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130. + action: + name: reject + + # GSI read routine QC (part-1) + # Exclude Type 254 with windComputationMethod==5 (clear-sky WV) --- obs tossed without passing to setup routine + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windNorthward + is_in: 254 + - variable: MetaData/windComputationMethod + is_in: 5 + action: + name: reject + + # Exclude data with satellite zenith angle > 68 for all types --- obs tossed without passing to setup routine + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: MetaData/satelliteZenithAngle + maxvalue: 68. + action: + name: reject + + # Exclude data over non-water surface type where latitude > 20N for Type 253 (IRLW) --- obs tossed and not passed to setup routine + # Notes: This check was missing, so added (eliu) + # Replace land_type_index_NPOSS with water_area_fraction (eliu) + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 253 + - variable: MetaData/latitude + minvalue: 20. + test variables: + - name: GeoVaLs/water_area_fraction + minvalue: 0.99 + action: + name: reject + + # GSI read routine QC (part-2) + # Reject obs with qualityInformationWithoutForecast < 85. OR > 100. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: MetaData/qualityInformationWithoutForecast + minvalue: 85. + maxvalue: 100. + action: + name: reject + + # GSI setupw routine QC + # Reject any ob Type [240–260] when pressure greater than 950 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240-260 + test variables: + - name: MetaData/pressure + maxvalue: 95001. + action: + name: reject + + # EUMETSAT IR (253) reject when pressure between 401 and 801 mb. + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/pressure + minvalue: 40101. + maxvalue: 80099. + - variable: ObsType/windEastward + is_in: 253 + action: + name: reject + + # EUMET VIS (243) reject when pressure less than 700 mb. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 243 + test variables: + - name: MetaData/pressure + minvalue: 70000. + action: + name: reject + + # EUMET WV (254) reject when pressure greater than 399 mb. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 254 + test variables: + - name: MetaData/pressure + maxvalue: 39900. + action: + name: reject + + # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation + # Turn this check off for now. + # Need to check if troposphere pressure was implemented correctly in fv3-jed + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + reference: GeoVaLs/tropopause_pressure + value: MetaData/pressure + minvalue: -5000. # 50 hPa above tropopause level, negative p-diff + action: + name: reject + + # All satwinds must adjust errors based on ObsErrorFactorPressureCheck + # prior to the SPDB check (i.e. the gross-error check). The gross-error + # check uses the adjusted errors for error-bound tightening and rejection, + # so this check has to come first. This check will inflate errors for obs + # that are too close to either the model top or bottom. + # Notes (eliu): GMAO added a required parameter: adjusted_error_name. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + surface_obs: false + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All satwinds subject to a gross-error check that contains significant + # modifiers for satwinds with a negative speed-bias. ALL wind gross-error + # checks are currently being done by the SatWindsSPDBCheck. + # CLEARED + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, + 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, + 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, + 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, + 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, + 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, + 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, + 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, + 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don"t want to inflate errors for duplication if one + # of the duplicates should be rejected. + # Notes (eliu): ObsErrorFactorDuplicateCheck obsfunction requires PreUseFlag (usage parameter from read_satwnd.f90). + # : Turn off duplicate check for now. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + # We are extending this to an additional filter that inflates final ob-errors across-the-board by + # 1/0.8 = 1.25. This is caused by the GSI value of nvqc being set to .true. in the global operational + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this nvqc functionality were to be switched off (i.e. if variational qc were to be turned off), + # you would want to remove this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + # End of Filters + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/scatwind_ascat_metop-a.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/scatwind_ascat_metop-a.yaml.j2 new file mode 100644 index 000000000..10cca6889 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/scatwind_ascat_metop-a.yaml.j2 @@ -0,0 +1,313 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: ascatw_ascat_metop-a + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + # Observation Operator + # -------------------- + obs operator: + name: VertInterp + # Use height vertical coordinate first + # vertical coordinate: geometric_height + vertical coordinate: geopotential_height + observation vertical coordinate group: DerivedVariables + observation vertical coordinate: adjustedHeight + interpolation method: linear + hofx scaling field: SurfaceWindScalingHeight + hofx scaling field group: DerivedVariables + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: VertInterp + vertical coordinate: geopotential_height + observation vertical coordinate group: DerivedVariables + observation vertical coordinate: adjustedHeight + interpolation method: linear + + # Observation Pre Filters (QC) + # ---------------------------- + obs pre filters: + - filter: Gaussian Thinning + horizontal_mesh: 75 + use_reduced_horizontal_grid: true + round_horizontal_bin_count_to_nearest: true + partition_longitude_bins_using_mesh: true + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Apply variable changes needed for rescaled height coordinate + - filter: Variable Transforms + Transform: AdjustedHeightCoordinate + SkipWhenNoObs: false + + # Apply variable changes needed for wind scaling + - filter: Variable Transforms + Transform: SurfaceWindScalingHeight + SkipWhenNoObs: false + + # Assign the initial observation error (constant value, 1.5 m/s right now). + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error parameter: 1.5 + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + # Reject all obs with PreQC mark already set above 3 + # NOTE: All scatwinds have an automatic PreQC mark of 2 (hard-wired default from GSI) + # - filter: PreQC + # maxvalue: 3 + # action: + # name: reject + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Reject all ASCAT (Type 290) winds with tsavg <= 273.0 (surface temperature) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 290 + # - variable: GeoVaLs/surface_temperature + - variable: GeoVaLs/surface_temperature_where_land + maxvalue: 273. + action: + name: reject + + # Reject all ASCAT (Type 290) winds with isflg /= 0 (non-water surface) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 290 + - variable: GeoVaLs/water_area_fraction + maxvalue: 0.99 + action: + name: reject + + # Reject ASCAT (Type 290) when observed component deviates from background by more than 5.0 m/s + # NOTE: This check can reject a u- or v-component of the same observation independently, which + # is fundamentally different from how GSI rejects obs (both components are rejected if + # either component fails a check). + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Arithmetic + options: + variables: + - name: ObsValue/windEastward + - name: HofX/windEastward + coefs: [1.0, -1.0] + minvalue: -5.0 + maxvalue: 5.0 + + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Arithmetic + options: + variables: + - name: ObsValue/windNorthward + - name: HofX/windNorthward + coefs: [1.0, -1.0] + minvalue: -5.0 + maxvalue: 5.0 + + # Reject OSCAT (Type 291) when observed component deviates from background by more than 6.0 m/s + # NOTE: This check can reject a u- or v-component of the same observation independently, which + # is fundamentally different from how GSI rejects obs (both components are rejected if + # either component fails a check). + - filter: Background Check + filter variables: + - name: windEastward + - name: windNorthward + threshold: 6. + absolute threshold: 6. + where: + - variable: ObsType/windEastward + is_in: 291 + action: + name: reject + + # Reject ASCAT (Type 290) when ambiguity check fails (returned value is negative) + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 290 + test variables: + - name: ObsFunction/ScatWindsAmbiguityCheck + options: + minimum_uv: 0.0001 # hard-coding a minimum-uv for transparancy, want this to basically be zero + maxvalue: 0. + action: + name: reject + + # All scatwinds must adjust errors based on ObsErrorFactorPressureCheck. + # This check will inflate errors for obs that are too close to either + # the model top or bottom. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 290-291 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 290-291 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All scatwinds subject to a gross error check. This is contained within + # the WindsSPDBCheck, although it is not exclusive to satwinds. + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [290, 291] + cgross: [5.0, 5.0] + error_min: [1.4, 1.4] + error_max: [6.1, 6.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [290, 291] + cgross: [5.0, 5.0] + error_min: [1.4, 1.4] + error_max: [6.1, 6.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don"t want to inflate errors for duplication if one + # of the duplicates should be rejected. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + # There is no across-the-board inflation for nvqc=.true. for scatwinds, presumably because for + # this inflation to take place both nvqc must be .true. AND ibeta must be >0, see: + # https://github.com/NOAA-EMC/GSI/blob/14ae595af1b03471287d322596d35c0665336e95/src/gsi/setupw.f90#L1229 + # GSI settings must have ibeta>0 for satwinds, but not for scatwinds. + # + # If the ibeta settings for scatwinds were to change while nvqc remained .true., we would extend YAML to + # an additional filter that inflates final ob-errors across-the-board by 1/0.8 = 1.25. NOTE: the nvqc setting + # is defaulted to .false. in GSI code, but is overridden in global operational configuration. See: + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this functionality were to be activated for scatwinds, you would want to include this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 290-291 + # action: + # name: inflate error + # inflation factor: 1.25 + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 290-291 + # action: + # name: inflate error + # inflation factor: 1.25 + # END OF FILTERS# + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/scatwind_ascat_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/scatwind_ascat_metop-b.yaml.j2 new file mode 100644 index 000000000..dfcfc6067 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/scatwind_ascat_metop-b.yaml.j2 @@ -0,0 +1,313 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: ascatw_ascat_metop-b + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + # Observation Operator + # -------------------- + obs operator: + name: VertInterp + # Use height vertical coordinate first + # vertical coordinate: geometric_height + vertical coordinate: geopotential_height + observation vertical coordinate group: DerivedVariables + observation vertical coordinate: adjustedHeight + interpolation method: linear + hofx scaling field: SurfaceWindScalingHeight + hofx scaling field group: DerivedVariables + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: VertInterp + vertical coordinate: geopotential_height + observation vertical coordinate group: DerivedVariables + observation vertical coordinate: adjustedHeight + interpolation method: linear + + # Observation Pre Filters (QC) + # ---------------------------- + obs pre filters: + - filter: Gaussian Thinning + horizontal_mesh: 75 + use_reduced_horizontal_grid: true + round_horizontal_bin_count_to_nearest: true + partition_longitude_bins_using_mesh: true + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Apply variable changes needed for rescaled height coordinate + - filter: Variable Transforms + Transform: AdjustedHeightCoordinate + SkipWhenNoObs: false + + # Apply variable changes needed for wind scaling + - filter: Variable Transforms + Transform: SurfaceWindScalingHeight + SkipWhenNoObs: false + + # Assign the initial observation error (constant value, 1.5 m/s right now). + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error parameter: 1.5 + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + # Reject all obs with PreQC mark already set above 3 + # NOTE: All scatwinds have an automatic PreQC mark of 2 (hard-wired default from GSI) + # - filter: PreQC + # maxvalue: 3 + # action: + # name: reject + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Reject all ASCAT (Type 290) winds with tsavg <= 273.0 (surface temperature) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 290 + # - variable: GeoVaLs/surface_temperature + - variable: GeoVaLs/surface_temperature_where_land + maxvalue: 273. + action: + name: reject + + # Reject all ASCAT (Type 290) winds with isflg /= 0 (non-water surface) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 290 + - variable: GeoVaLs/water_area_fraction + maxvalue: 0.99 + action: + name: reject + + # Reject ASCAT (Type 290) when observed component deviates from background by more than 5.0 m/s + # NOTE: This check can reject a u- or v-component of the same observation independently, which + # is fundamentally different from how GSI rejects obs (both components are rejected if + # either component fails a check). + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Arithmetic + options: + variables: + - name: ObsValue/windEastward + - name: HofX/windEastward + coefs: [1.0, -1.0] + minvalue: -5.0 + maxvalue: 5.0 + + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Arithmetic + options: + variables: + - name: ObsValue/windNorthward + - name: HofX/windNorthward + coefs: [1.0, -1.0] + minvalue: -5.0 + maxvalue: 5.0 + + # Reject OSCAT (Type 291) when observed component deviates from background by more than 6.0 m/s + # NOTE: This check can reject a u- or v-component of the same observation independently, which + # is fundamentally different from how GSI rejects obs (both components are rejected if + # either component fails a check). + - filter: Background Check + filter variables: + - name: windEastward + - name: windNorthward + threshold: 6. + absolute threshold: 6. + where: + - variable: ObsType/windEastward + is_in: 291 + action: + name: reject + + # Reject ASCAT (Type 290) when ambiguity check fails (returned value is negative) + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 290 + test variables: + - name: ObsFunction/ScatWindsAmbiguityCheck + options: + minimum_uv: 0.0001 # hard-coding a minimum-uv for transparancy, want this to basically be zero + maxvalue: 0. + action: + name: reject + + # All scatwinds must adjust errors based on ObsErrorFactorPressureCheck. + # This check will inflate errors for obs that are too close to either + # the model top or bottom. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 290-291 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 290-291 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All scatwinds subject to a gross error check. This is contained within + # the WindsSPDBCheck, although it is not exclusive to satwinds. + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [290, 291] + cgross: [5.0, 5.0] + error_min: [1.4, 1.4] + error_max: [6.1, 6.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [290, 291] + cgross: [5.0, 5.0] + error_min: [1.4, 1.4] + error_max: [6.1, 6.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don"t want to inflate errors for duplication if one + # of the duplicates should be rejected. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + # There is no across-the-board inflation for nvqc=.true. for scatwinds, presumably because for + # this inflation to take place both nvqc must be .true. AND ibeta must be >0, see: + # https://github.com/NOAA-EMC/GSI/blob/14ae595af1b03471287d322596d35c0665336e95/src/gsi/setupw.f90#L1229 + # GSI settings must have ibeta>0 for satwinds, but not for scatwinds. + # + # If the ibeta settings for scatwinds were to change while nvqc remained .true., we would extend YAML to + # an additional filter that inflates final ob-errors across-the-board by 1/0.8 = 1.25. NOTE: the nvqc setting + # is defaulted to .false. in GSI code, but is overridden in global operational configuration. See: + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this functionality were to be activated for scatwinds, you would want to include this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 290-291 + # action: + # name: inflate error + # inflation factor: 1.25 + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 290-291 + # action: + # name: inflate error + # inflation factor: 1.25 + # END OF FILTERS# + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/sfc.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/sfc.yaml.j2 new file mode 100644 index 000000000..3192c2acd --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/sfc.yaml.j2 @@ -0,0 +1,40 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: sfc + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [stationPressure] + + # Observation Operator + # -------------------- + obs operator: + name: SfcPCorrected + da_psfc_scheme: GSI + geovar_sfc_geomz: surface_geometric_height + geovar_geomz: geopotential_height + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: Identity + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/sfcship.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/sfcship.yaml.j2 new file mode 100644 index 000000000..2c5e25e12 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/sfcship.yaml.j2 @@ -0,0 +1,100 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: sfcship + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + overwrite: true + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [stationPressure, airTemperature, specificHumidity] + + # Observation Operator + # -------------------- + obs operator: + name: Composite + components: + - name: VertInterp + variables: + - name: airTemperature + - name: specificHumidity + - name: SfcPCorrected + variables: + - name: stationPressure + da_psfc_scheme: GSI + geovar_sfc_geomz: surface_geometric_height + geovar_geomz: geopotential_height + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: Identity + + # Observation Filters (QC) + # ------------------------ + obs filters: + # Observation range sanity check + - filter: Bounds Check + filter variables: + - name: stationPressure + minvalue: 37499.0 + maxvalue: 106999.0 + action: + name: reject + - filter: Bounds Check + filter variables: + - name: airTemperature + minvalue: 195.0 + maxvalue: 327.0 + action: + name: reject + - filter: Bounds Check + filter variables: + - name: specificHumidity + minvalue: 0.0 + maxvalue: 0.03499 + action: + name: reject + + # Gross error check with (O - B) / ObsError greater than threshold. + - filter: Background Check + filter variables: + - name: stationPressure + threshold: 3.6 + absolute threshold: 990.0 + action: + name: reject + defer to post: true + - filter: Background Check + filter variables: + - name: airTemperature + threshold: 7.0 + absolute threshold: 9.0 + action: + name: reject + defer to post: true + + # Reject all ObsType 183 + - filter: BlackList + where: + - variable: + name: ObsType/stationPressure + is_in: 183 + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 new file mode 100644 index 000000000..f56414440 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 @@ -0,0 +1,128 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: sondes + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsgrouping: + group variables: ["stationIdentification"] + sort variable: "pressure" + sort order: "descending" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [stationPressure, airTemperature, windEastward, windNorthward, + specificHumidity] + + # Observation Operator + # -------------------- + obs operator: + name: Composite + components: + - name: VertInterp + variables: + - name: airTemperature + - name: windEastward + - name: windNorthward + - name: specificHumidity + - name: SfcPCorrected + da_psfc_scheme: GSI + geovar_sfc_geomz: surface_geopotential_height + geovar_geomz: geopotential_height + variables: + - name: stationPressure + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: Composite + components: + - name: VertInterp + variables: + - name: airTemperature + - name: windEastward + - name: windNorthward + - name: specificHumidity + - name: Identity + variables: + - name: stationPressure + + # Observation Filters (QC) + # ------------------------ + obs filters: + # + # Reject all obs with PreQC mark already set above 3 + - filter: PreQC + maxvalue: 3.0 + action: + name: reject + # + # Observation Range Sanity Check + - filter: Bounds Check + filter variables: + - name: stationPressure + minvalue: 37499.0 + maxvalue: 106999.0 + action: + name: reject + # + # Assign obsError + - filter: Perform Action + filter variables: + - name: stationPressure + action: + name: assign error + error parameter: 100.0 # 1.0 hPa + # + # Assign the initial observation error, based on height/pressure + - filter: Perform Action + filter variables: + - name: stationPressure + action: + name: assign error + error function: + name: ObsErrorModelStepwiseLinear@ObsFunction + options: + xvar: + name: ObsValue/stationPressure + xvals: [80000.0, 75000.0] + errors: [110.0, 120.0] # 1.1 mb below 800 mb and 1.2 mb agove 750 mb + # + - filter: Perform Action + filter variables: + - name: stationPressure + action: + name: inflate error + inflation variable: + name: ObsErrorFactorSfcPressure@ObsFunction + options: + error_min: 100.0 # 1 mb + error_max: 300.0 # 3 mb + geovar_geomz: geopotential_height + geovar_sfc_geomz: surface_geopotential_height + # + # Gross error check with (O - B) / ObsError greater than threshold + - filter: Background Check + filter variables: + - name: stationPressure + threshold: 3.6 + absolute threshold: 990.0 + action: + name: reject + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 new file mode 100644 index 000000000..c89cb2767 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 @@ -0,0 +1,177 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: SSMIS F17 + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + simulated variables: [brightness_temperature] + channels: {{ get_satellite_variable(observation_from_jcb, "simulated") }} + + # Observation Operator + # -------------------- + obs operator: + name: CRTM + Absorbers: [H2O, O3, CO2] + obs options: + Sensor_ID: &{{observation_from_jcb}}_sensor_id ssmis_f17 + EndianType: little_endian + CoefficientPath: "{{crtm_coefficient_path}}" + + # Observation Bias Correction (VarBC) + # ----------------------------------- + obs bias: + input file: "{{atm_obsbiasin_path}}/{{atm_obsbiasin_prefix}}{{observation_from_jcb}}{{atm_obsbiasin_suffix}}" + output file: "{{atm_obsbiasout_path}}/{{atm_obsbiasout_prefix}}{{observation_from_jcb}}{{atm_obsbiasout_suffix}}" + variational bc: + predictors: + - name: constant + - name: cloud_liquid_water + sensor: SSMIS + ch19h: 12 + ch19v: 13 + ch22v: 14 + ch37h: 15 + ch37v: 16 + ch91v: 17 + ch91h: 18 + - name: cosine_of_latitude_times_orbit_node + - name: sine_of_latitude + - name: lapse_rate + order: 2 + tlapse: &{{observation_from_jcb}}_tlapse "{{atm_obsbiasin_path}}/{{atm_obstlapsein_prefix}}{{observation_from_jcb}}{{atm_obstlapsein_suffix}}" + - name: lapse_rate + tlapse: *{{observation_from_jcb}}_tlapse + - name: emissivity + - name: scan_angle + var_name: scan_position + order: 4 + - name: scan_angle + var_name: scan_position + order: 3 + - name: scan_angle + var_name: scan_position + order: 2 + - name: scan_angle + var_name: scan_position + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + - filter: Perform Action + filter variables: + - name: brightness_temperature + channels: 1-24 + action: + name: assign error + error parameter vector: [1.5, 0.5, 0.5, 0.5, 0.5, 1, 1, 3, 3, 3, 3, 2.4, 1.27, + 1.44, 3, 1.34, 1.74, 3.75, 3, 3, 2, 6.4, 1, 1] + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + #step1: Gross check (setuprad) + - filter: Background Check + filter variables: + - name: brightness_temperature + channels: 1-24 + threshold: 1.5 + action: + name: reject + #step1: Gross check(qcmod) + - filter: Background Check + filter variables: + - name: brightness_temperature + channels: 1-24 + absolute threshold: 3.5 + remove bias correction: true + action: + name: reject + + - filter: Difference Check + filter variables: + - name: brightness_temperature + channels: 1-2,12-16 + reference: brightness_temperature_2@ObsValue + value: brightness_temperature_2@HofX + minvalue: -1.5 + maxvalue: 1.5 + where: + - variable: + name: water_area_fraction@GeoVaLs + maxvalue: 0.99 + #QC_terrain: If seviri and terrain height > 2km. do not use + - filter: Domain Check + filter variables: + - name: brightness_temperature + channels: 1-24 + where: + - variable: + name: height_above_mean_sea_level@MetaData + maxvalue: 2000.0 + #do not use over mixed surface + - filter: BlackList + filter variables: + - name: brightness_temperature + channels: 1-3,8-18 + where: + - variable: + name: land_area_fraction@GeoVaLs + maxvalue: 0.99 + - variable: + name: water_area_fraction@GeoVaLs + maxvalue: 0.99 + - variable: + name: ice_area_fraction@GeoVaLs + maxvalue: 0.99 + - variable: + name: surface_snow_area_fraction@GeoVaLs + maxvalue: 0.99 + #step4: Generate q.c. bounds and modified variances + - filter: BlackList + filter variables: + - name: brightness_temperature + channels: 1-24 + action: + name: inflate error + inflation variable: + # Surface Jacobian check + name: ObsErrorFactorSurfJacobianRad@ObsFunction + channels: 1-24 + options: + channels: 1-24 + sensor: *{{observation_from_jcb}}_sensor_id + obserr_demisf: [0.010, 0.010, 0.010, 0.010, 0.010] + obserr_dtempf: [0.500, 0.500, 0.500, 0.500, 0.500] + # Useflag Check + - filter: Bounds Check + filter variables: + - name: brightness_temperature + channels: 1-24 + test variables: + - name: ChannelUseflagCheckRad@ObsFunction + channels: 1-24 + options: + channels: 1-24 + use_flag: [1, -1, -1, -1, 1, 1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 1] + minvalue: 1.0e-12 + action: + name: reject + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 new file mode 100644 index 000000000..5cd86df70 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 @@ -0,0 +1,177 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: SSMIS F18 + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + simulated variables: [brightness_temperature] + channels: {{ get_satellite_variable(observation_from_jcb, "simulated") }} + + # Observation Operator + # -------------------- + obs operator: + name: CRTM + Absorbers: [H2O, O3, CO2] + obs options: + Sensor_ID: &{{observation_from_jcb}}_sensor_id ssmis_f17 + EndianType: little_endian + CoefficientPath: "{{crtm_coefficient_path}}" + + # Observation Bias Correction (VarBC) + # ----------------------------------- + obs bias: + input file: "{{atm_obsbiasin_path}}/{{atm_obsbiasin_prefix}}{{observation_from_jcb}}{{atm_obsbiasin_suffix}}" + output file: "{{atm_obsbiasout_path}}/{{atm_obsbiasout_prefix}}{{observation_from_jcb}}{{atm_obsbiasout_suffix}}" + variational bc: + predictors: + - name: constant + - name: cloud_liquid_water + sensor: SSMIS + ch19h: 12 + ch19v: 13 + ch22v: 14 + ch37h: 15 + ch37v: 16 + ch91v: 17 + ch91h: 18 + - name: cosine_of_latitude_times_orbit_node + - name: sine_of_latitude + - name: lapse_rate + order: 2 + tlapse: &{{observation_from_jcb}}_tlapse "{{atm_obsbiasin_path}}/{{atm_obstlapsein_prefix}}{{observation_from_jcb}}{{atm_obstlapsein_suffix}}" + - name: lapse_rate + tlapse: *{{observation_from_jcb}}_tlapse + - name: emissivity + - name: scan_angle + var_name: scan_position + order: 4 + - name: scan_angle + var_name: scan_position + order: 3 + - name: scan_angle + var_name: scan_position + order: 2 + - name: scan_angle + var_name: scan_position + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + - filter: Perform Action + filter variables: + - name: brightness_temperature + channels: 1-24 + action: + name: assign error + error parameter vector: [1.5, 0.5, 0.5, 0.5, 0.5, 1, 1, 3, 3, 3, 3, 2.4, 1.27, + 1.44, 3, 1.34, 1.74, 3.75, 3, 3, 2, 6.4, 1, 1] + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + #step1: Gross check (setuprad) + - filter: Background Check + filter variables: + - name: brightness_temperature + channels: 1-24 + threshold: 1.5 + action: + name: reject + #step1: Gross check(qcmod) + - filter: Background Check + filter variables: + - name: brightness_temperature + channels: 1-24 + absolute threshold: 3.5 + remove bias correction: true + action: + name: reject + + - filter: Difference Check + filter variables: + - name: brightness_temperature + channels: 1-2,12-16 + reference: brightness_temperature_2@ObsValue + value: brightness_temperature_2@HofX + minvalue: -1.5 + maxvalue: 1.5 + where: + - variable: + name: water_area_fraction@GeoVaLs + maxvalue: 0.99 + #QC_terrain: If seviri and terrain height > 2km. do not use + - filter: Domain Check + filter variables: + - name: brightness_temperature + channels: 1-24 + where: + - variable: + name: height_above_mean_sea_level@MetaData + maxvalue: 2000.0 + #do not use over mixed surface + - filter: BlackList + filter variables: + - name: brightness_temperature + channels: 1-3,8-18 + where: + - variable: + name: land_area_fraction@GeoVaLs + maxvalue: 0.99 + - variable: + name: water_area_fraction@GeoVaLs + maxvalue: 0.99 + - variable: + name: ice_area_fraction@GeoVaLs + maxvalue: 0.99 + - variable: + name: surface_snow_area_fraction@GeoVaLs + maxvalue: 0.99 + #step4: Generate q.c. bounds and modified variances + - filter: BlackList + filter variables: + - name: brightness_temperature + channels: 1-24 + action: + name: inflate error + inflation variable: + # Surface Jacobian check + name: ObsErrorFactorSurfJacobianRad@ObsFunction + channels: 1-24 + options: + channels: 1-24 + sensor: *{{observation_from_jcb}}_sensor_id + obserr_demisf: [0.010, 0.010, 0.010, 0.010, 0.010] + obserr_dtempf: [0.500, 0.500, 0.500, 0.500, 0.500] + # Useflag Check + - filter: Bounds Check + filter variables: + - name: brightness_temperature + channels: 1-24 + test variables: + - name: ChannelUseflagCheckRad@ObsFunction + channels: 1-24 + options: + channels: 1-24 + use_flag: [1, -1, -1, -1, 1, 1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 1] + minvalue: 1.0e-12 + action: + name: reject + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 From 81426fc3634b6cf953937ec0d11355459cbb39c7 Mon Sep 17 00:00:00 2001 From: danholdaway Date: Tue, 2 Apr 2024 16:43:23 -0400 Subject: [PATCH 016/199] cris npp chronicle following jeff w --- .../atmosphere/cris-fsr_npp.yaml | 1284 ++++++++++++++--- 1 file changed, 1089 insertions(+), 195 deletions(-) diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/cris-fsr_npp.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/cris-fsr_npp.yaml index 638bcef33..ad7b83cd6 100644 --- a/parm/jcb-gdas/observation_chronicle/atmosphere/cris-fsr_npp.yaml +++ b/parm/jcb-gdas/observation_chronicle/atmosphere/cris-fsr_npp.yaml @@ -1,6 +1,7 @@ # Instrument metadata # ------------------- commissioned: 2017-10-20T00:00:00 +# decommissioned: 2021-05-22T00:00:00 (Effectively decommissioned since no active channels.) observer_type: satellite # Type of chronicle to use @@ -20,174 +21,174 @@ channel_variables: iaerosol: min channel_values: 19: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 24: [ 1, -1, 1, 0.700, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] - 26: [ 1, -1, 1, 0.700, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 24: [ 1, 1, 1, 0.700, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 26: [ 1, 1, 1, 0.700, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 27: [ 1, -1, 1, 0.700, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 28: [ 1, -1, 1, 0.700, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 28: [ 1, 1, 1, 0.700, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 31: [ 1, -1, 1, 1.359, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 32: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 32: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 33: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 37: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] - 39: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 42: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] - 44: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 47: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] - 49: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 37: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 39: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 42: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 44: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 47: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 49: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 50: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 51: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 51: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 52: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 53: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 53: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 54: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 55: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 55: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 56: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 57: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 57: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 58: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 59: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 59: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 60: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 61: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 61: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 62: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 63: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 63: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 64: [ 1, -1, 1, 0.756, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 65: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 65: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 66: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 67: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 67: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 68: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 69: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 69: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 70: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 71: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 71: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 72: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 73: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 73: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 74: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 75: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 75: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 76: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 77: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 77: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 78: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 79: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 79: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 80: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 81: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 81: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 82: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 83: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 83: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 84: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 85: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 85: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 86: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 87: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 87: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 88: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 89: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 89: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 90: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 91: [ 1, -1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 91: [ 1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 92: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 93: [ 1, -1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 93: [ 1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 94: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 95: [ 1, -1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 95: [ 1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 96: [ 1, -1, 1, 0.635, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 97: [ 1, -1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 97: [ 1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 98: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 99: [ 1, -1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 99: [ 1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 100: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 101: [ 1, -1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 102: [ 1, -1, 1, 0.735, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 103: [ 1, -1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 103: [ 1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 104: [ 1, -1, 1, 0.878, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 105: [ 1, -1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 105: [ 1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 106: [ 1, -1, 1, 0.696, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 107: [ 1, -1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 107: [ 1, 1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 108: [ 1, -1, 1, 2.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 109: [ 1, -1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 109: [ 1, 1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 110: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 111: [ 1, -1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 111: [ 1, 1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 112: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 113: [ 1, -1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 113: [ 1, 1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 114: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 115: [ 1, -1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 115: [ 1, 1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 116: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 117: [ 1, -1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 117: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 118: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 119: [ 1, -1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 119: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 120: [ 1, -1, 1, 0.701, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 121: [ 1, -1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 121: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 122: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 123: [ 1, -1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 123: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 124: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 125: [ 1, -1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 125: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 126: [ 1, -1, 1, 0.663, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 127: [ 1, -1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 127: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 128: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 129: [ 1, -1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 129: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 130: [ 1, -1, 1, 1.083, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 131: [ 1, -1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 131: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 132: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 133: [ 1, -1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 133: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 134: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 135: [ 1, -1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 135: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 136: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 137: [ 1, -1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 137: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 138: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 139: [ 1, -1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 139: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 140: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 141: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 141: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 142: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 143: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 143: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 144: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 145: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 145: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 146: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 147: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 147: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 148: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 149: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 149: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 150: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 151: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 151: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 152: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 153: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 153: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 154: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 155: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 155: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 156: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 157: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 157: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 158: [ 1, -1, 1, 0.773, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 159: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 159: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 160: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 161: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 162: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 163: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 163: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 164: [ 1, -1, 1, 0.813, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 165: [ 1, -1, 1, 0.907, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 166: [ 1, -1, 1, 0.802, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 167: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 167: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 168: [ 1, -1, 1, 1.493, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 169: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 170: [ 1, -1, 1, 0.856, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 171: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 171: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 172: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 173: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 174: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 175: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 175: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 176: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 177: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 178: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 179: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 179: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 180: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 181: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 182: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 183: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 183: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 184: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 185: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 186: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 187: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 187: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 188: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 189: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 190: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 190: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 191: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 192: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 193: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 194: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 194: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 195: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 196: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 197: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 197: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 198: [ 1, -1, 1, 2.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 199: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 200: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 200: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 208: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 211: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 211: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 216: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 224: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 224: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 234: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 236: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 238: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] @@ -199,24 +200,24 @@ channel_values: 264: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 266: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 268: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 275: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 279: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 275: [ 1, 1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 279: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 283: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 285: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 291: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 291: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 295: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 301: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 305: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 311: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] - 332: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 342: [ 1, -1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] - 389: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 311: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 332: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 342: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 389: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 400: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 402: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 404: [ 1, -1, 1, 2.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 406: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 410: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] - 427: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 410: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 427: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 439: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 440: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 441: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] @@ -225,19 +226,19 @@ channel_values: 455: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 458: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 461: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 464: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 464: [ 1, 1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 467: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 470: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 473: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 475: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 482: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 482: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 486: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 487: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 490: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 493: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 496: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 499: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 501: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 501: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 503: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 505: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 511: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] @@ -247,7 +248,7 @@ channel_values: 519: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 520: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 522: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 529: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 529: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] 534: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 563: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 568: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] @@ -280,111 +281,111 @@ channel_values: 694: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 700: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 707: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 710: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] - 713: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] - 714: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 718: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 720: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 722: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 725: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 728: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 735: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 742: [ 0, -1, 0, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 748: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 753: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 762: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 780: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 784: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 798: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 849: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 860: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 862: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 866: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 874: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 882: [ 0, -1, 0, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] - 890: [ 0, -1, 0, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] - 898: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 906: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 907: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 908: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 914: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 937: [ 0, -1, 0, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] - 972: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 973: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 978: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 980: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 981: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 988: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 995: [ 0, -1, 0, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] - 998: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1000: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1003: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1008: [ 0, -1, 0, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] - 1009: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1010: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1014: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1017: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1018: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1020: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1022: [ 0, -1, 0, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] - 1024: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1026: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1029: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1030: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1032: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1034: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1037: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1038: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1041: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1042: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1044: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1046: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1049: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1050: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1053: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1054: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1058: [ 0, -1, 0, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] - 1060: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1062: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1064: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1066: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1069: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1076: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1077: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1080: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1086: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1091: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1095: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1101: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1109: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1112: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1121: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1128: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1133: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1163: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1172: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1187: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1189: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1205: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1211: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1219: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1231: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1245: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1271: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1289: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1300: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1313: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1316: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1325: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1329: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1346: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1347: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1473: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1474: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1491: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1499: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1553: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] - 1570: [ 0, -1, 0, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 710: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 713: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 714: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 718: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 720: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 722: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 725: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 728: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 735: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 742: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 748: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 753: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 762: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 780: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 784: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 798: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 849: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 860: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 862: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 866: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 874: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 882: [ 1, 1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 890: [ 1, 1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 898: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 906: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 907: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 908: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 914: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 937: [ 1, 1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 972: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 973: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 978: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 980: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 981: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 988: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 995: [ 1, 1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 998: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1000: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1003: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1008: [ 1, 1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 1009: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1010: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1014: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1017: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1018: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1020: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1022: [ 1, 1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 1024: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1026: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1029: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1030: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1032: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1034: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1037: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1038: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1041: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1042: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1044: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1046: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1049: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1050: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1053: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1054: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1058: [ 1, 1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 1060: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1062: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1064: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1066: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1069: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1076: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1077: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1080: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1086: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1091: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1095: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1101: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1109: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1112: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1121: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1128: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1133: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1163: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1172: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1187: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1189: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1205: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1211: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1219: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1231: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1245: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1271: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1289: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1300: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1313: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1316: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1325: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1329: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1346: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1347: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1473: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1474: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1491: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1499: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1553: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1570: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 1596: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 1602: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 1619: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] @@ -450,3 +451,896 @@ channel_values: 2171: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 2175: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 2182: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + + +# Chronicle of changes for this instrument +# ---------------------------------------- +chronicle: + +- action_date: "2019-03-26T00:00:00" + justification: 'The CrIS instrument on NPP developed anomalies starting on 18Z of March 27, 2019.' + adjust_variable_for_all_channels: + variables: [active] + values: [-1] + +- action_date: "2019-04-16T18:00:00" + justification: 'NPP out because of mid-wave failure 3/25/19 18z. LW/SW data returns 4/16/19 18z but mid-wave channels missing' + channel_values: + 19: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 24: [ 1, 1, 1, 0.700, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 26: [ 1, 1, 1, 0.700, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 27: [ 1, -1, 1, 0.700, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 28: [ 1, 1, 1, 0.700, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 31: [ 1, -1, 1, 1.359, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 32: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 33: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 37: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 39: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 42: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 44: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 47: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 49: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 50: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 51: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 52: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 53: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 54: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 55: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 56: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 57: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 58: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 59: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 60: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 61: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 62: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 63: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 64: [ 1, -1, 1, 0.756, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 65: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 66: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 67: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 68: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 69: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 70: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 71: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 72: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 73: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 74: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 75: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 76: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 77: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 78: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 79: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 80: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 81: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 82: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 83: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 84: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 85: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 86: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 87: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 88: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 89: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 90: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 91: [ 1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 92: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 93: [ 1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 94: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 95: [ 1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 96: [ 1, -1, 1, 0.635, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 97: [ 1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 98: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 99: [ 1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 100: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 101: [ 1, -1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 102: [ 1, -1, 1, 0.735, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 103: [ 1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 104: [ 1, -1, 1, 0.878, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 105: [ 1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 106: [ 1, -1, 1, 0.696, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 107: [ 1, 1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 108: [ 1, -1, 1, 2.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 109: [ 1, 1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 110: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 111: [ 1, 1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 112: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 113: [ 1, 1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 114: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 115: [ 1, 1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 116: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 117: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 118: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 119: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 120: [ 1, -1, 1, 0.701, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 121: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 122: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 123: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 124: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 125: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 126: [ 1, -1, 1, 0.663, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 127: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 128: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 129: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 130: [ 1, -1, 1, 1.083, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 131: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 132: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 133: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 134: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 135: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 136: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 137: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 138: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 139: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 140: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 141: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 142: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 143: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 144: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 145: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 146: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 147: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 148: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 149: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 150: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 151: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 152: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 153: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 154: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 155: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 156: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 157: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 158: [ 1, -1, 1, 0.773, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 159: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 160: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 161: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 162: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 163: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 164: [ 1, -1, 1, 0.813, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 165: [ 1, -1, 1, 0.907, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 166: [ 1, -1, 1, 0.802, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 167: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 168: [ 1, -1, 1, 1.493, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 169: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 170: [ 1, -1, 1, 0.856, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 171: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 172: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 173: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 174: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 175: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 176: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 177: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 178: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 179: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 180: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 181: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 182: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 183: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 184: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 185: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 186: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 187: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 188: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 189: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 190: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 191: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 192: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 193: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 194: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 195: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 196: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 197: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 198: [ 1, -1, 1, 2.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 199: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 200: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 208: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 211: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 216: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 224: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 234: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 236: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 238: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 239: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 242: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 246: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 248: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 255: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 264: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 266: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 268: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 275: [ 1, 1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 279: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 283: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 285: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 291: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 295: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 301: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 305: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 311: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 332: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 342: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 389: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 400: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 402: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 404: [ 1, -1, 1, 2.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 406: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 410: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 427: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 439: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 440: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 441: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 445: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 449: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 455: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 458: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 461: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 464: [ 1, 1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 467: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 470: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 473: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 475: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 482: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 486: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 487: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 490: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 493: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 496: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 499: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 501: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 503: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 505: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 511: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 513: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 514: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 518: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 519: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 520: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 522: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 529: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 534: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 563: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 568: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 575: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 592: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 594: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 596: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 598: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 600: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 602: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 604: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 611: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 614: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 616: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 618: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 620: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 622: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 626: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 631: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 638: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 646: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 648: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 652: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 659: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 673: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 675: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 678: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 684: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 688: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 694: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 700: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 707: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 710: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 713: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 714: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 718: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 720: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 722: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 725: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 728: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 735: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 742: [ 1, -1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 748: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 753: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 762: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 780: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 784: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 798: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 849: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 860: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 862: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 866: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 874: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 882: [ 1, -1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 890: [ 1, -1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 898: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 906: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 907: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 908: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 914: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 937: [ 1, -1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 972: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 973: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 978: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 980: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 981: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 988: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 995: [ 1, -1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 998: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1000: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1003: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1008: [ 1, -1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 1009: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1010: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1014: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1017: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1018: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1020: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1022: [ 1, -1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 1024: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1026: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1029: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1030: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1032: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1034: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1037: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1038: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1041: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1042: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1044: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1046: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1049: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1050: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1053: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1054: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1058: [ 1, -1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 1060: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1062: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1064: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1066: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1069: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1076: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1077: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1080: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1086: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1091: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1095: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1101: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1109: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1112: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1121: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1128: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1133: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1163: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1172: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1187: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1189: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1205: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1211: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1219: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1231: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1245: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1271: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1289: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1300: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1313: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1316: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1325: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1329: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1346: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1347: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1473: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1474: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1491: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1499: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1553: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1570: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1596: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1602: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1619: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1624: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1635: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1939: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1940: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1941: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1942: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1943: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1944: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1945: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1946: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1947: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1948: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1949: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1950: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1951: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1952: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1953: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1954: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1955: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1956: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1957: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1958: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1959: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1960: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1961: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1962: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1963: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1964: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1965: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1966: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1967: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1968: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1969: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1970: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1971: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1972: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1973: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1974: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1975: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1976: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1977: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1978: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1979: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1980: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1981: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1982: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1983: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1984: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1985: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1986: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1987: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2119: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2140: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2143: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2147: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2153: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2158: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2161: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2168: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2171: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2175: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2182: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + +- action_date: "2019-06-24T18:00:00" + justification: 'Switch to side-2 electronics after 6/24/19 12z. Initially passive for a few days to adjust to side-2' + adjust_variable_for_all_channels: + variables: [active] + values: [-1] + +- action_date: "2019-04-16T18:00:00" + justification: 'NPP out because of mid-wave failure 3/25/19 18z. LW/SW data returns 4/16/19 18z but mid-wave channels missing' + channel_values: + 19: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 24: [1, 1, 1, 0.700, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 26: [1, 1, 1, 0.700, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 27: [1, -1, 1, 0.700, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 28: [1, 1, 1, 0.700, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 31: [1, -1, 1, 1.359, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 32: [1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 33: [1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 37: [1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 39: [1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 42: [1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 44: [1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 47: [1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 49: [1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 50: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 51: [1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 52: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 53: [1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 54: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 55: [1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 56: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 57: [1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 58: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 59: [1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 60: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 61: [1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 62: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 63: [1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 64: [1, -1, 1, 0.756, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 65: [1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 66: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 67: [1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 68: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 69: [1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 70: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 71: [1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 72: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 73: [1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 74: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 75: [1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 76: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 77: [1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 78: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 79: [1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 80: [1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 81: [1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 82: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 83: [1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 84: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 85: [1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 86: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 87: [1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 88: [1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 89: [1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 90: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 91: [1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 92: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 93: [1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 94: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 95: [1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 96: [1, -1, 1, 0.635, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 97: [1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 98: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 99: [1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 100: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 101: [1, -1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 102: [1, -1, 1, 0.735, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 103: [1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 104: [1, -1, 1, 0.878, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 105: [1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 106: [1, -1, 1, 0.696, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 107: [1, 1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 108: [1, -1, 1, 2.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 109: [1, 1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 110: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 111: [1, 1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 112: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 113: [1, 1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 114: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 115: [1, 1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 116: [1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 117: [1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 118: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 119: [1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 120: [1, -1, 1, 0.701, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 121: [1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 122: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 123: [1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 124: [1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 125: [1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 126: [1, -1, 1, 0.663, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 127: [1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 128: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 129: [1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 130: [1, -1, 1, 1.083, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 131: [1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 132: [1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 133: [1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 134: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 135: [1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 136: [1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 137: [1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 138: [1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 139: [1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 140: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 141: [1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 142: [1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 143: [1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 144: [1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 145: [1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 146: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 147: [1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 148: [1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 149: [1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 150: [1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 151: [1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 152: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 153: [1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 154: [1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 155: [1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 156: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 157: [1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 158: [1, -1, 1, 0.773, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 159: [1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 160: [1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 161: [1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 162: [1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 163: [1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 164: [1, -1, 1, 0.813, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 165: [1, -1, 1, 0.907, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 166: [1, -1, 1, 0.802, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 167: [1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 168: [1, -1, 1, 1.493, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 169: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 170: [1, -1, 1, 0.856, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 171: [1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 172: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 173: [1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 174: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 175: [1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 176: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 177: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 178: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 179: [1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 180: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 181: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 182: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 183: [1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 184: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 185: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 186: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 187: [1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 188: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 189: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 190: [1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 191: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 192: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 193: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 194: [1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 195: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 196: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 197: [1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 198: [1, -1, 1, 2.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 199: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 200: [1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 208: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 211: [1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 216: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 224: [1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 234: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 236: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 238: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 239: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 242: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 246: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 248: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 255: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 264: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 266: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 268: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 275: [1, 1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 279: [1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 283: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 285: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 291: [1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 295: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 301: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 305: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 311: [1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 332: [1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 342: [1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 389: [1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 400: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 402: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 404: [1, -1, 1, 2.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 406: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 410: [1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 427: [1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 439: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 440: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 441: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 445: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 449: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 455: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 458: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 461: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 464: [1, 1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 467: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 470: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 473: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 475: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 482: [1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 486: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 487: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 490: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 493: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 496: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 499: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 501: [1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 503: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 505: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 511: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 513: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 514: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 518: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 519: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 520: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 522: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 529: [1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 534: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 563: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 568: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 575: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 592: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 594: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 596: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 598: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 600: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 602: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 604: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 611: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 614: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 616: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 618: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 620: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 622: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 626: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 631: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 638: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 646: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 648: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 652: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 659: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 673: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 675: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 678: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 684: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 688: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 694: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 700: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 707: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 710: [1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 713: [1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 714: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 718: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 720: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 722: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 725: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 728: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 735: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 742: [1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 748: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 753: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 762: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 780: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 784: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 798: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 849: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 860: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 862: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 866: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 874: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 882: [1, 1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 890: [1, 1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 898: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 906: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 907: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 908: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 914: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 937: [1, 1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 972: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 973: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 978: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 980: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 981: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 988: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 995: [1, 1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 998: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1000: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1003: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1008: [1, 1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 1009: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1010: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1014: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1017: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1018: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1020: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1022: [1, 1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 1024: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1026: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1029: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1030: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1032: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1034: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1037: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1038: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1041: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1042: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1044: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1046: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1049: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1050: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1053: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1054: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1058: [1, 1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 1060: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1062: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1064: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1066: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1069: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1076: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1077: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1080: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1086: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1091: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1095: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1101: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1109: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1112: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1121: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1128: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1133: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1163: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1172: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1187: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1189: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1205: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1211: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1219: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1231: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1245: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1271: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1289: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1300: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1313: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1316: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1325: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1329: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1346: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1347: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1473: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1474: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1491: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1499: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1553: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1570: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1596: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1602: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1619: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1624: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1635: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1939: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1940: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1941: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1942: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1943: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1944: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1945: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1946: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1947: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1948: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1949: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1950: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1951: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1952: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1953: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1954: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1955: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1956: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1957: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1958: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1959: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1960: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1961: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1962: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1963: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1964: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1965: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1966: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1967: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1968: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1969: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1970: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1971: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1972: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1973: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1974: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1975: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1976: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1977: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1978: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1979: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1980: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1981: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1982: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1983: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1984: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1985: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1986: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1987: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2119: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2140: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2143: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2147: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2153: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2158: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2161: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2168: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2171: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2175: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2182: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + +- action_date: "2019-06-24T18:00:00" + justification: 'Side 2 long-wave failure. Never turned back on for GFS.' + adjust_variable_for_all_channels: + variables: [active] + values: [-1] From 00cc73efe22e19565f7a75077ed5098f58261b66 Mon Sep 17 00:00:00 2001 From: danholdaway Date: Fri, 5 Apr 2024 15:40:59 +0000 Subject: [PATCH 017/199] channels should not be a list --- parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 index 63119f938..9511ecf3a 100644 --- a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 @@ -335,7 +335,7 @@ options: channels: *{{observation_from_jcb}}_simulated_channels sensor: *{{observation_from_jcb}}_sensor_id - use_flag: &{{observation_from_jcb}}_active_channels {{ get_satellite_variable(observation_from_jcb, "active") }} + use_flag: &{{observation_from_jcb}}_active_channels [{{ get_satellite_variable(observation_from_jcb, "active") }}] maxvalue: 1.0e-12 action: name: reject From 37796366a24e2f096d65e12fef281e30e84c6da7 Mon Sep 17 00:00:00 2001 From: danholdaway Date: Fri, 5 Apr 2024 15:15:58 -0400 Subject: [PATCH 018/199] add ps to background vars --- parm/jcb-gdas/model/atmosphere/background.yaml.j2 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/parm/jcb-gdas/model/atmosphere/background.yaml.j2 b/parm/jcb-gdas/model/atmosphere/background.yaml.j2 index 66d902073..af8b509c3 100644 --- a/parm/jcb-gdas/model/atmosphere/background.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/background.yaml.j2 @@ -7,6 +7,6 @@ filename_trcr: "%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc" filename_sfcd: "%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc" filename_sfcw: "%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc" filename_cplr: "%yyyy%mm%dd.%hh%MM%ss.coupler.res" -state variables: [ua,va,t,delp,sphum,ice_wat,liq_wat,o3mr,phis, +state variables: [ua,va,t,delp,ps,sphum,ice_wat,liq_wat,o3mr,phis, slmsk,sheleg,tsea,vtype,stype,vfrac,stc,smc,snwdph, u_srf,v_srf,f10m] From 73b43c1dc1df84ad1fab3979fb47ab398cad83cc Mon Sep 17 00:00:00 2001 From: danholdaway Date: Fri, 5 Apr 2024 16:34:03 -0400 Subject: [PATCH 019/199] some chronicle fixes --- parm/jcb-gdas/.gitignore | 1 + .../atmosphere/amsua_n19.yaml | 49 +++++++-------- .../atmosphere/cris-fsr_npp.yaml | 8 +-- .../observations/atmosphere/amsua_n19.yaml.j2 | 60 +++++++------------ 4 files changed, 48 insertions(+), 70 deletions(-) create mode 100644 parm/jcb-gdas/.gitignore diff --git a/parm/jcb-gdas/.gitignore b/parm/jcb-gdas/.gitignore new file mode 100644 index 000000000..5509140f2 --- /dev/null +++ b/parm/jcb-gdas/.gitignore @@ -0,0 +1 @@ +*.DS_Store diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/amsua_n19.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/amsua_n19.yaml index 5a8e3ab4a..10be79b57 100644 --- a/parm/jcb-gdas/observation_chronicle/atmosphere/amsua_n19.yaml +++ b/parm/jcb-gdas/observation_chronicle/atmosphere/amsua_n19.yaml @@ -1,7 +1,6 @@ # Instrument metadata # ------------------- -commissioned: 2009-04-14T00:00:00 - +commissioned: 2009-12-15T00:00:00 observer_type: satellite # Type of chronicle to use # Instrument initial configuration @@ -13,38 +12,36 @@ channel_variables: error: max error_cld: max ermax: max - var_b: min - var_pg: min - icld_det: min - icloud: min - iaerosol: min + x0: max + x1: max channel_values: - 1: [ 1, 1, 1, 2.50, 20.00, 4.50, 10.00, 0.00, -2, 1, -1 ] - 2: [ 1, 1, 1, 2.20, 18.00, 4.50, 10.00, 0.00, -2, 1, -1 ] - 3: [ 1, 1, 1, 2.00, 12.00, 4.50, 10.00, 0.00, -2, 1, -1 ] - 4: [ 1, 1, 1, 0.55, 3.00, 2.50, 10.00, 0.00, -2, 1, -1 ] - 5: [ 1, 1, 1, 0.30, 0.50, 2.00, 10.00, 0.00, -2, 1, -1 ] - 6: [ 1, 1, 1, 0.23, 0.30, 2.00, 10.00, 0.00, -2, 1, -1 ] - 7: [ 1, 1, 1, 0.23, 0.23, 2.00, 10.00, 0.00, -2, 1, -1 ] - 8: [ 1, 1, 1, 0.25, 0.25, 2.00, 10.00, 0.00, -2, 1, -1 ] - 9: [ 1, 1, 1, 0.25, 0.25, 2.00, 10.00, 0.00, -2, 1, -1 ] - 10: [ 1, 1, 1, 0.35, 0.35, 2.00, 10.00, 0.00, -2, 1, -1 ] - 11: [ 1, 1, 1, 0.40, 0.40, 2.50, 10.00, 0.00, -2, 1, -1 ] - 12: [ 1, 1, 1, 0.55, 0.55, 3.50, 10.00, 0.00, -2, 1, -1 ] - 13: [ 1, 1, 1, 0.80, 0.80, 4.50, 10.00, 0.00, -2, 1, -1 ] - 14: [ 1, 1, 0, 4.00, 4.00, 4.50, 10.00, 0.00, -2, 1, -1 ] - 15: [ 1, 1, 1, 3.50, 18.00, 4.50, 10.00, 0.00, -2, 1, -1 ] +# sim act bcd err errcld ermax x0 x1 + 1: [ 1, 1, 1, 2.50, 20.00, 4.50, 0.050, 0.600 ] + 2: [ 1, 1, 1, 2.20, 18.00, 4.50, 0.030, 0.450 ] + 3: [ 1, 1, 1, 2.00, 12.00, 4.50, 0.030, 0.400 ] + 4: [ 1, 1, 1, 0.55, 3.00, 2.50, 0.020, 0.450 ] + 5: [ 1, 1, 1, 0.30, 0.50, 2.00, 0.000, 1.000 ] + 6: [ 1, 1, 1, 0.23, 0.30, 2.00, 0.100, 1.500 ] + 7: [ 1, 1, 1, 0.23, 0.23, 2.00, 0.000, 0.000 ] + 8: [ 1, 1, 1, 0.25, 0.25, 2.00, 0.000, 0.000 ] + 9: [ 1, 1, 1, 0.25, 0.25, 2.00, 0.000, 0.000 ] + 10: [ 1, 1, 1, 0.35, 0.35, 2.00, 0.000, 0.000 ] + 11: [ 1, 1, 1, 0.40, 0.40, 2.50, 0.000, 0.000 ] + 12: [ 1, 1, 1, 0.55, 0.55, 3.50, 0.000, 0.000 ] + 13: [ 1, 1, 1, 0.80, 0.80, 4.50, 0.000, 0.000 ] + 14: [ 1, 1, 0, 4.00, 4.00, 4.50, 0.000, 0.000 ] # No bias correction of Ch.14 + 15: [ 1, 1, 1, 3.50, 18.00, 4.50, 0.030, 0.200 ] # Chronicle of changes for this instrument # ---------------------------------------- -chronicle: +chronicles: - action_date: "2009-12-22T00:00:00" justification: 'Ch8 noisy (noise started on 12/21/2009). Switch to monitoring only' channel_values: - 8: [ 1, -1, 1, 0.25, 0.25, 2.00, 10.00, 0.00, -2, 1, -1 ] + 8: [ 1, -1, 1, 0.25, 0.25, 2.00, 0.000, 0.000 ] -- action_date: "2009-12-22T00:00:00" +- action_date: "2014-01-29T00:00:00" justification: 'Per NCEP r35918. Ch 7 switch to monitoring only' channel_values: - 7: [ 1, -1, 1, 0.23, 0.23, 2.00, 10.00, 0.00, -2, 1, -1 ] + 7: [ 1, -1, 1, 0.23, 0.23, 2.00, 0.000, 0.000 ] diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/cris-fsr_npp.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/cris-fsr_npp.yaml index ad7b83cd6..26d4a0a43 100644 --- a/parm/jcb-gdas/observation_chronicle/atmosphere/cris-fsr_npp.yaml +++ b/parm/jcb-gdas/observation_chronicle/atmosphere/cris-fsr_npp.yaml @@ -1,6 +1,6 @@ # Instrument metadata # ------------------- -commissioned: 2017-10-20T00:00:00 +commissioned: "2018-02-12T12:00:00" # decommissioned: 2021-05-22T00:00:00 (Effectively decommissioned since no active channels.) observer_type: satellite # Type of chronicle to use @@ -455,7 +455,7 @@ channel_values: # Chronicle of changes for this instrument # ---------------------------------------- -chronicle: +chronicles: - action_date: "2019-03-26T00:00:00" justification: 'The CrIS instrument on NPP developed anomalies starting on 18Z of March 27, 2019.' @@ -904,7 +904,7 @@ chronicle: variables: [active] values: [-1] -- action_date: "2019-04-16T18:00:00" +- action_date: "2019-06-26T18:00:00" justification: 'NPP out because of mid-wave failure 3/25/19 18z. LW/SW data returns 4/16/19 18z but mid-wave channels missing' channel_values: 19: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] @@ -1339,7 +1339,7 @@ chronicle: 2175: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] 2182: [1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] -- action_date: "2019-06-24T18:00:00" +- action_date: "2021-05-22T18:00:00" justification: 'Side 2 long-wave failure. Never turned back on for GFS.' adjust_variable_for_all_channels: variables: [active] diff --git a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 index 9511ecf3a..244a1cd58 100644 --- a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 @@ -82,14 +82,10 @@ clwret_ch238: 1 clwret_ch314: 2 clwret_types: [ObsValue, HofX] - x0: [0.050, 0.030, 0.030, 0.020, 0.000, 0.100, 0.000, 0.000, 0.000, 0.000, - 0.000, 0.000, 0.000, 0.000, 0.030] - x1: [0.600, 0.450, 0.400, 0.450, 1.000, 1.500, 0.000, 0.000, 0.000, 0.000, - 0.000, 0.000, 0.000, 0.000, 0.200] - err0: [2.500, 2.200, 2.000, 0.550, 0.300, 0.230, 0.230, 0.250, 0.250, 0.350, - 0.400, 0.550, 0.800, 3.000, 3.500] - err1: [20.000, 18.000, 12.000, 3.000, 0.500, 0.300, 0.230, 0.250, 0.250, - 0.350, 0.400, 0.550, 0.800, 3.000, 18.000] + x0: &{{observation_from_jcb}}_x0 [{{ get_satellite_variable(observation_from_jcb, "x0") }}] + x1: &{{observation_from_jcb}}_x1 [{{ get_satellite_variable(observation_from_jcb, "x1") }}] + err0: &{{observation_from_jcb}}_err0 [{{ get_satellite_variable(observation_from_jcb, "error") }}] + err1: &{{observation_from_jcb}}_err1 [{{ get_satellite_variable(observation_from_jcb, "error_cld") }}] # CLW Retrieval Check - filter: Bounds Check filter variables: @@ -128,8 +124,7 @@ channels: *{{observation_from_jcb}}_simulated_channels options: channels: *{{observation_from_jcb}}_simulated_channels - obserr_clearsky: [2.500, 2.200, 2.000, 0.550, 0.300, 0.230, 0.230, 0.250, - 0.250, 0.350, 0.400, 0.550, 0.800, 3.000, 3.500] + obserr_clearsky: *{{observation_from_jcb}}_err0 clwret_function: name: ObsFunction/CLWRetMW options: @@ -147,14 +142,10 @@ clwret_ch238: 1 clwret_ch314: 2 clwret_types: [ObsValue, HofX] - x0: [0.050, 0.030, 0.030, 0.020, 0.000, 0.100, 0.000, 0.000, 0.000, 0.000, - 0.000, 0.000, 0.000, 0.000, 0.030] - x1: [0.600, 0.450, 0.400, 0.450, 1.000, 1.500, 0.000, 0.000, 0.000, 0.000, - 0.000, 0.000, 0.000, 0.000, 0.200] - err0: [2.500, 2.200, 2.000, 0.550, 0.300, 0.230, 0.230, 0.250, 0.250, - 0.350, 0.400, 0.550, 0.800, 3.000, 3.500] - err1: [20.000, 18.000, 12.000, 3.000, 0.500, 0.300, 0.230, 0.250, 0.250, - 0.350, 0.400, 0.550, 0.800, 3.000, 18.000] + x0: *{{observation_from_jcb}}_x0 + x1: *{{observation_from_jcb}}_x1 + err0: *{{observation_from_jcb}}_err0 + err1: *{{observation_from_jcb}}_err1 maxvalue: 0.0 action: name: reject @@ -251,8 +242,7 @@ clwret_ch314: 2 clwret_types: [HofX] bias_application: HofX - clwret_clearsky: [0.050, 0.030, 0.030, 0.020, 0.000, 0.100, 0.000, 0.000, - 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.030] + clwret_clearsky: *{{observation_from_jcb}}_x0 obserr_function: name: ObsFunction/ObsErrorModelRamp channels: *{{observation_from_jcb}}_simulated_channels @@ -264,16 +254,11 @@ clwret_ch238: 1 clwret_ch314: 2 clwret_types: [ObsValue, HofX] - x0: [0.050, 0.030, 0.030, 0.020, 0.000, 0.100, 0.000, 0.000, 0.000, - 0.000, 0.000, 0.000, 0.000, 0.000, 0.030] - x1: [0.600, 0.450, 0.400, 0.450, 1.000, 1.500, 0.000, 0.000, 0.000, - 0.000, 0.000, 0.000, 0.000, 0.000, 0.200] - err0: [2.500, 2.200, 2.000, 0.550, 0.300, 0.230, 0.230, 0.250, 0.250, - 0.350, 0.400, 0.550, 0.800, 3.000, 3.500] - err1: [20.000, 18.000, 12.000, 3.000, 0.500, 0.300, 0.230, 0.250, 0.250, - 0.350, 0.400, 0.550, 0.800, 3.000, 18.000] - obserr_clearsky: [2.500, 2.200, 2.000, 0.550, 0.300, 0.230, 0.230, 0.250, - 0.250, 0.350, 0.400, 0.550, 0.800, 3.000, 3.500] + x0: *{{observation_from_jcb}}_x0 + x1: *{{observation_from_jcb}}_x1 + err0: *{{observation_from_jcb}}_err0 + err1: *{{observation_from_jcb}}_err1 + obserr_clearsky: *{{observation_from_jcb}}_err0 # Gross check - filter: Background Check filter variables: @@ -312,16 +297,11 @@ clwret_ch314: 2 clwret_types: [ObsValue, HofX] bias_application: HofX - x0: [0.050, 0.030, 0.030, 0.020, 0.000, 0.100, 0.000, 0.000, 0.000, 0.000, - 0.000, 0.000, 0.000, 0.000, 0.030] - x1: [0.600, 0.450, 0.400, 0.450, 1.000, 1.500, 0.000, 0.000, 0.000, 0.000, - 0.000, 0.000, 0.000, 0.000, 0.200] - err0: [2.500, 2.200, 2.000, 0.550, 0.300, 0.230, 0.230, 0.250, 0.250, - 0.350, 0.400, 0.550, 0.800, 3.000, 3.500] - err1: [20.000, 18.000, 12.000, 3.000, 0.500, 0.300, 0.230, 0.250, 0.250, - 0.350, 0.400, 0.550, 0.800, 3.000, 18.000] - obserr_bound_max: [4.5, 4.5, 4.5, 2.5, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.5, - 3.5, 4.5, 4.5, 4.5] + x0: *{{observation_from_jcb}}_x0 + x1: *{{observation_from_jcb}}_x1 + err0: *{{observation_from_jcb}}_err0 + err1: *{{observation_from_jcb}}_err1 + obserr_bound_max: [{{ get_satellite_variable(observation_from_jcb, "ermax") }}] action: name: reject # Inter-channel check From a99527492d02b460cf3b08cdfd73d2093014fd42 Mon Sep 17 00:00:00 2001 From: danholdaway Date: Mon, 8 Apr 2024 10:30:54 -0400 Subject: [PATCH 020/199] add option for writing increment to cubed sphere --- .../final_increment_cubed_sphere.yaml.j2 | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 parm/jcb-gdas/model/atmosphere/final_increment_cubed_sphere.yaml.j2 diff --git a/parm/jcb-gdas/model/atmosphere/final_increment_cubed_sphere.yaml.j2 b/parm/jcb-gdas/model/atmosphere/final_increment_cubed_sphere.yaml.j2 new file mode 100644 index 000000000..301f24b89 --- /dev/null +++ b/parm/jcb-gdas/model/atmosphere/final_increment_cubed_sphere.yaml.j2 @@ -0,0 +1,18 @@ +output: + state component: + filetype: cube sphere history + filename: ./anl/atminc.%yyyy%mm%dd_%hh%MM%ssz.nc4 + provider: ufs + fields to write: [ugrd,vgrd,tmp,pressfc,spfh,icmr,clwmr,o3mr] +geometry: + fms initialization: + namelist filename: "{{fv3jedi_files_path}}/fmsmpp.nml" + field table filename: "{{fv3jedi_files_path}}/field_table" + akbk: "{{fv3jedi_files_path}}/akbk.nc4" + layout: + - {{layout_x}} + - {{layout_y}} + npx: {{npx_anl}} + npy: {{npy_anl}} + npz: {{npz_anl}} + field metadata override: "{{fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" From 3f3cad4beb939a324d40199f9072581f2c02a389 Mon Sep 17 00:00:00 2001 From: danholdaway Date: Tue, 9 Apr 2024 19:57:36 +0000 Subject: [PATCH 021/199] no chronicle for letkf --- parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 index 48c601276..94fd7a83c 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 @@ -18,7 +18,7 @@ io pool: max pool size: 1 simulated variables: [brightnessTemperature] - channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_simulated_channels(observation_from_jcb) }} + channels: &{{observation_from_jcb}}_simulated_channels 4-6,9-14 # Observation Operator # -------------------- From 0c4af081455cf085507ed0bd32ef81ecd19cd6f7 Mon Sep 17 00:00:00 2001 From: danholdaway Date: Mon, 15 Apr 2024 18:52:57 +0000 Subject: [PATCH 022/199] add avhrr satwind yamls --- .../atmosphere/satwind_avhrr_n15.yaml.j2 | 289 ++++++++++++++++++ .../atmosphere/satwind_avhrr_n18.yaml.j2 | 289 ++++++++++++++++++ .../atmosphere/satwind_avhrr_n19.yaml.j2 | 289 ++++++++++++++++++ 3 files changed, 867 insertions(+) create mode 100644 parm/jcb-gdas/observations/atmosphere/satwind_avhrr_n15.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/satwind_avhrr_n18.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/satwind_avhrr_n19.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/satwind_avhrr_n15.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwind_avhrr_n15.yaml.j2 new file mode 100644 index 000000000..aa8247854 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/satwind_avhrr_n15.yaml.j2 @@ -0,0 +1,289 @@ +- obs space: + name: satwind_avhrr_n15 + obsdatain: + engine: + type: H5File + obsfile: obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + obs operator: + name: VertInterp + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + linear obs operator: + name: VertInterp + + # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI's thinning of AHI/Himawari-8 satwinds + # results in more JEDI satwinds in the diag file than in GSI, but far fewer JEDI satwinds assimilated than + # GSI. JEDI under-counts assimilated winds by roughly 25-40%, relative to GSI, and this under-count is not + # even including the temporal thinning which is applied in GSI but not JEDI (by this filter below). See + # GDASApp Issue #741 for details: https://github.com/NOAA-EMC/GDASApp/issues/741 + #obs pre filters: + #- filter: Gaussian Thinning + # horizontal_mesh: 200 + # vertical_mesh: 10000 + # use_reduced_horizontal_grid: true + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: False + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + + obs post filters: + # Assign the initial observation error, based on height/pressure + # Hard-wiring to prepobs_errtable.global by Type + # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML + + # Type 244 (AVHRR LWIR) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 244 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000.,105000.,100000.,95000.,90000.,85000.,80000.,75000.,70000.,65000.,60000.,55000.,50000.,45000.,40000.,35000.,30000.,25000.,20000.,15000.,10000.,7500.,5000.,4000.,3000.,2000.,1000.,500.,400.,300.,200.,100.,0.] #Pressure (Pa) + errors: [3.8,3.8,3.8,3.8,3.8,3.8,3.8,3.8,3.9,3.9,4.,4.,4.1,5.,6.,6.3,6.6,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.] + + # sanity-check criteria + # Observation Range Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130. + maxvalue: 130. + action: + name: reject + + # Velocity Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130. + action: + name: reject + + # Reject any observation with a /=0 surface type (non-water surface) within + # 200 hPa of the surface pressure (as part of the LNVD check). + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + reference: GeoVaLs/surface_pressure + value: MetaData/pressure + maxvalue: -20000. # within 200 hPa above surface pressure, negative p-diff + action: + name: reject + + # LNVD check + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/SatWindsLNVDCheck + maxvalue: 3. + action: + name: reject + + # GSI setupw routine QC + # Reject any ob Type [240–260] when pressure greater than 950 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240-260 + test variables: + - name: MetaData/pressure + maxvalue: 95001. + action: + name: reject + + # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation + # Turn this check off for now. + # Need to check if troposphere pressure was implemented correctly in fv3-jed + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + reference: GeoVaLs/tropopause_pressure + value: MetaData/pressure + minvalue: -5000. # 50 hPa above tropopause level, negative p-diff + action: + name: reject + + # All satwinds must adjust errors based on ObsErrorFactorPressureCheck + # prior to the SPDB check (i.e. the gross-error check). The gross-error + # check uses the adjusted errors for error-bound tightening and rejection, + # so this check has to come first. This check will inflate errors for obs + # that are too close to either the model top or bottom. + # Notes (eliu): GMAO added a required parameter: adjusted_error_name. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + surface_obs: false + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All satwinds subject to a gross-error check that contains significant + # modifiers for satwinds with a negative speed-bias. ALL wind gross-error + # checks are currently being done by the SatWindsSPDBCheck. + # CLEARED + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [ 2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [ 2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don't want to inflate errors for duplication if one + # of the duplicates should be rejected. + # Notes (eliu): ObsErrorFactorDuplicateCheck obsfunction requires PreUseFlag (usage parameter from read_satwnd.f90). + # : Turn off duplicate check for now. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + + # We are extending this to an additional filter that inflates final ob-errors across-the-board by + # 1/0.8 = 1.25. This is caused by the GSI value of nvqc being set to .true. in the global operational + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this nvqc functionality were to be switched off (i.e. if variational qc were to be turned off), + # you would want to remove this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + + # End of Filters diff --git a/parm/jcb-gdas/observations/atmosphere/satwind_avhrr_n18.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwind_avhrr_n18.yaml.j2 new file mode 100644 index 000000000..fee0d851f --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/satwind_avhrr_n18.yaml.j2 @@ -0,0 +1,289 @@ +- obs space: + name: satwind_avhrr_n18 + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + obs operator: + name: VertInterp + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + linear obs operator: + name: VertInterp + + # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI's thinning of AHI/Himawari-8 satwinds + # results in more JEDI satwinds in the diag file than in GSI, but far fewer JEDI satwinds assimilated than + # GSI. JEDI under-counts assimilated winds by roughly 25-40%, relative to GSI, and this under-count is not + # even including the temporal thinning which is applied in GSI but not JEDI (by this filter below). See + # GDASApp Issue #741 for details: https://github.com/NOAA-EMC/GDASApp/issues/741 + #obs pre filters: + #- filter: Gaussian Thinning + # horizontal_mesh: 200 + # vertical_mesh: 10000 + # use_reduced_horizontal_grid: true + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: False + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + + obs post filters: + # Assign the initial observation error, based on height/pressure + # Hard-wiring to prepobs_errtable.global by Type + # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML + + # Type 244 (AVHRR LWIR) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 244 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000.,105000.,100000.,95000.,90000.,85000.,80000.,75000.,70000.,65000.,60000.,55000.,50000.,45000.,40000.,35000.,30000.,25000.,20000.,15000.,10000.,7500.,5000.,4000.,3000.,2000.,1000.,500.,400.,300.,200.,100.,0.] #Pressure (Pa) + errors: [3.8,3.8,3.8,3.8,3.8,3.8,3.8,3.8,3.9,3.9,4.,4.,4.1,5.,6.,6.3,6.6,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.] + + # sanity-check criteria + # Observation Range Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130. + maxvalue: 130. + action: + name: reject + + # Velocity Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130. + action: + name: reject + + # Reject any observation with a /=0 surface type (non-water surface) within + # 200 hPa of the surface pressure (as part of the LNVD check). + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + reference: GeoVaLs/surface_pressure + value: MetaData/pressure + maxvalue: -20000. # within 200 hPa above surface pressure, negative p-diff + action: + name: reject + + # LNVD check + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/SatWindsLNVDCheck + maxvalue: 3. + action: + name: reject + + # GSI setupw routine QC + # Reject any ob Type [240–260] when pressure greater than 950 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240-260 + test variables: + - name: MetaData/pressure + maxvalue: 95001. + action: + name: reject + + # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation + # Turn this check off for now. + # Need to check if troposphere pressure was implemented correctly in fv3-jed + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + reference: GeoVaLs/tropopause_pressure + value: MetaData/pressure + minvalue: -5000. # 50 hPa above tropopause level, negative p-diff + action: + name: reject + + # All satwinds must adjust errors based on ObsErrorFactorPressureCheck + # prior to the SPDB check (i.e. the gross-error check). The gross-error + # check uses the adjusted errors for error-bound tightening and rejection, + # so this check has to come first. This check will inflate errors for obs + # that are too close to either the model top or bottom. + # Notes (eliu): GMAO added a required parameter: adjusted_error_name. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + surface_obs: false + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All satwinds subject to a gross-error check that contains significant + # modifiers for satwinds with a negative speed-bias. ALL wind gross-error + # checks are currently being done by the SatWindsSPDBCheck. + # CLEARED + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [ 2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [ 2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don't want to inflate errors for duplication if one + # of the duplicates should be rejected. + # Notes (eliu): ObsErrorFactorDuplicateCheck obsfunction requires PreUseFlag (usage parameter from read_satwnd.f90). + # : Turn off duplicate check for now. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + + # We are extending this to an additional filter that inflates final ob-errors across-the-board by + # 1/0.8 = 1.25. This is caused by the GSI value of nvqc being set to .true. in the global operational + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this nvqc functionality were to be switched off (i.e. if variational qc were to be turned off), + # you would want to remove this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + + # End of Filters diff --git a/parm/jcb-gdas/observations/atmosphere/satwind_avhrr_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwind_avhrr_n19.yaml.j2 new file mode 100644 index 000000000..6af10f99e --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/satwind_avhrr_n19.yaml.j2 @@ -0,0 +1,289 @@ +- obs space: + name: satwind_avhrr_n19 + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + obs operator: + name: VertInterp + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + linear obs operator: + name: VertInterp + + # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI's thinning of AHI/Himawari-8 satwinds + # results in more JEDI satwinds in the diag file than in GSI, but far fewer JEDI satwinds assimilated than + # GSI. JEDI under-counts assimilated winds by roughly 25-40%, relative to GSI, and this under-count is not + # even including the temporal thinning which is applied in GSI but not JEDI (by this filter below). See + # GDASApp Issue #741 for details: https://github.com/NOAA-EMC/GDASApp/issues/741 + #obs pre filters: + #- filter: Gaussian Thinning + # horizontal_mesh: 200 + # vertical_mesh: 10000 + # use_reduced_horizontal_grid: true + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: False + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + + obs post filters: + # Assign the initial observation error, based on height/pressure + # Hard-wiring to prepobs_errtable.global by Type + # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML + + # Type 244 (AVHRR LWIR) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 244 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000.,105000.,100000.,95000.,90000.,85000.,80000.,75000.,70000.,65000.,60000.,55000.,50000.,45000.,40000.,35000.,30000.,25000.,20000.,15000.,10000.,7500.,5000.,4000.,3000.,2000.,1000.,500.,400.,300.,200.,100.,0.] #Pressure (Pa) + errors: [3.8,3.8,3.8,3.8,3.8,3.8,3.8,3.8,3.9,3.9,4.,4.,4.1,5.,6.,6.3,6.6,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.] + + # sanity-check criteria + # Observation Range Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130. + maxvalue: 130. + action: + name: reject + + # Velocity Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130. + action: + name: reject + + # Reject any observation with a /=0 surface type (non-water surface) within + # 200 hPa of the surface pressure (as part of the LNVD check). + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + reference: GeoVaLs/surface_pressure + value: MetaData/pressure + maxvalue: -20000. # within 200 hPa above surface pressure, negative p-diff + action: + name: reject + + # LNVD check + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/SatWindsLNVDCheck + maxvalue: 3. + action: + name: reject + + # GSI setupw routine QC + # Reject any ob Type [240–260] when pressure greater than 950 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240-260 + test variables: + - name: MetaData/pressure + maxvalue: 95001. + action: + name: reject + + # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation + # Turn this check off for now. + # Need to check if troposphere pressure was implemented correctly in fv3-jed + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + reference: GeoVaLs/tropopause_pressure + value: MetaData/pressure + minvalue: -5000. # 50 hPa above tropopause level, negative p-diff + action: + name: reject + + # All satwinds must adjust errors based on ObsErrorFactorPressureCheck + # prior to the SPDB check (i.e. the gross-error check). The gross-error + # check uses the adjusted errors for error-bound tightening and rejection, + # so this check has to come first. This check will inflate errors for obs + # that are too close to either the model top or bottom. + # Notes (eliu): GMAO added a required parameter: adjusted_error_name. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + surface_obs: false + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All satwinds subject to a gross-error check that contains significant + # modifiers for satwinds with a negative speed-bias. ALL wind gross-error + # checks are currently being done by the SatWindsSPDBCheck. + # CLEARED + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [ 2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [ 2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don't want to inflate errors for duplication if one + # of the duplicates should be rejected. + # Notes (eliu): ObsErrorFactorDuplicateCheck obsfunction requires PreUseFlag (usage parameter from read_satwnd.f90). + # : Turn off duplicate check for now. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + + # We are extending this to an additional filter that inflates final ob-errors across-the-board by + # 1/0.8 = 1.25. This is caused by the GSI value of nvqc being set to .true. in the global operational + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this nvqc functionality were to be switched off (i.e. if variational qc were to be turned off), + # you would want to remove this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + + # End of Filters From 48a272df629cd41705d904b1b75eb4025149b6ce Mon Sep 17 00:00:00 2001 From: danholdaway Date: Wed, 24 Apr 2024 20:49:38 +0000 Subject: [PATCH 023/199] Add the increment util --- .../atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 | 62 +++++++++++++++++++ .../fv3jedi_fv3inc_variational.yaml.j2 | 62 +++++++++++++++++++ .../final_increment_cubed_sphere.yaml.j2 | 2 +- 3 files changed, 125 insertions(+), 1 deletion(-) create mode 100644 parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 create mode 100644 parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 new file mode 100644 index 000000000..e07592e4f --- /dev/null +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 @@ -0,0 +1,62 @@ +variable change: + variable change name: Model2GeoVaLs + input variables: &bkgvars [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr,surface_geopotential_height] + output variables: &fv3incrvars [ua,va,t,sphum,ice_wat,liq_wat,o3mr,delp,hydrostatic_delz] +jedi increment variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] +fv3 increment variables: *fv3incrvars +background geometry: + fms initialization: + namelist filename: ./fv3jedi/fmsmpp.nml + field table filename: ./fv3jedi/field_table + akbk: ./fv3jedi/akbk.nc4 + layout: + - {{ layout_x }} + - {{ layout_y }} + npx: {{ npx_ges }} + npy: {{ npy_ges }} + npz: {{ npz_ges }} + field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_history.yaml +jedi increment geometry: + fms initialization: + namelist filename: ./fv3jedi/fmsmpp.nml + field table filename: ./fv3jedi/field_table + akbk: ./fv3jedi/akbk.nc4 + layout: + - {{ layout_x }} + - {{ layout_y }} + npx: {{ npx_ges }} + npy: {{ npy_ges }} + npz: {{ npz_ges }} + field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_history.yaml +fv3 increment geometry: + fms initialization: + namelist filename: ./fv3jedi/fmsmpp.nml + field table filename: ./fv3jedi/field_table + akbk: ./fv3jedi/akbk.nc4 + layout: + - {{ layout_x }} + - {{ layout_y }} + npx: {{ npx_ges }} + npy: {{ npy_ges }} + npz: {{ npz_ges }} + field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_fv3inc.yaml +members from template: + template: + background input: + datetime: '{{ atm_background_time_iso }}' + filetype: cube sphere history + provider: ufs + datapath: ./bkg/mem%mem% + filename: {{ EPREFIX }}atmf006.nc + state variables: *bkgvars + jedi increment input: + filetype: cube sphere history + filename: ./anl/mem%mem%/atminc.%yyyy%mm%dd.%hh%MM%ssz.nc4 + provider: ufs + fv3 increment output: + filetype: auxgrid + gridtype: gaussian + filename: ./anl/mem%mem%/atminc. + pattern: '%mem%' + nmembers: {{ number_ensemble_members }} + zero padding: 3 diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 new file mode 100644 index 000000000..c6fa86557 --- /dev/null +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 @@ -0,0 +1,62 @@ +variable change: + variable change name: Model2GeoVaLs + input variables: &bkgvars [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr,phis] + output variables: &fv3incrvars [ua,va,t,sphum,ice_wat,liq_wat,o3mr,delp,hydrostatic_delz] +jedi increment variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] +fv3 increment variables: *fv3incrvars +background geometry: + fms initialization: + namelist filename: ./fv3jedi/fmsmpp.nml + field table filename: ./fv3jedi/field_table + akbk: ./fv3jedi/akbk.nc4 + layout: + - {{ layout_x }} + - {{ layout_y }} + npx: {{ npx_ges }} + npy: {{ npy_ges }} + npz: {{ npz_ges }} + field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_restart.yaml +jedi increment geometry: + fms initialization: + namelist filename: ./fv3jedi/fmsmpp.nml + field table filename: ./fv3jedi/field_table + akbk: ./fv3jedi/akbk.nc4 + layout: + - {{ layout_x }} + - {{ layout_y }} + npx: {{ npx_anl }} + npy: {{ npy_anl }} + npz: {{ npz_anl }} + field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_history.yaml +fv3 increment geometry: + fms initialization: + namelist filename: ./fv3jedi/fmsmpp.nml + field table filename: ./fv3jedi/field_table + akbk: ./fv3jedi/akbk.nc4 + layout: + - {{ layout_x }} + - {{ layout_y }} + npx: {{ npx_anl }} + npy: {{ npy_anl }} + npz: {{ npz_anl }} + field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_fv3inc.yaml +members: +- background input: + datapath: ./bkg + filetype: fms restart + datetime: "{{ atm_background_time_iso }}" + filename is datetime templated: true + filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" + filename_trcr: "%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc" + filename_sfcd: "%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc" + filename_sfcw: "%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc" + filename_cplr: "%yyyy%mm%dd.%hh%MM%ss.coupler.res" + state variables: *bkgvars + jedi increment input: + filetype: cube sphere history + filename: ./anl/atminc.%yyyy%mm%dd_%hh%MM%ssz.nc4 + provider: ufs + fv3 increment output: + filetype: auxgrid + gridtype: gaussian + filename: ./anl/atminc. diff --git a/parm/jcb-gdas/model/atmosphere/final_increment_cubed_sphere.yaml.j2 b/parm/jcb-gdas/model/atmosphere/final_increment_cubed_sphere.yaml.j2 index 301f24b89..a95cef127 100644 --- a/parm/jcb-gdas/model/atmosphere/final_increment_cubed_sphere.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/final_increment_cubed_sphere.yaml.j2 @@ -15,4 +15,4 @@ geometry: npx: {{npx_anl}} npy: {{npy_anl}} npz: {{npz_anl}} - field metadata override: "{{fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" + field metadata override: "{{fv3jedi_files_path}}/fv3jedi_fieldmetadata_history.yaml" From 9050efc936acbb6cdb9989ae71e1be996852c342 Mon Sep 17 00:00:00 2001 From: danholdaway Date: Wed, 24 Apr 2024 20:55:10 +0000 Subject: [PATCH 024/199] satwind bug --- parm/jcb-gdas/observations/atmosphere/satwind_avhrr_n15.yaml.j2 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/parm/jcb-gdas/observations/atmosphere/satwind_avhrr_n15.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwind_avhrr_n15.yaml.j2 index aa8247854..7fe70cdd3 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwind_avhrr_n15.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwind_avhrr_n15.yaml.j2 @@ -3,7 +3,7 @@ obsdatain: engine: type: H5File - obsfile: obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" obsdataout: engine: type: H5File From f9c8576cea4fea9ee5a5d2953d5f8c53c3f4ab23 Mon Sep 17 00:00:00 2001 From: RussTreadon-NOAA Date: Mon, 29 Apr 2024 17:40:01 +0000 Subject: [PATCH 025/199] minor adjustments required by g-w JEDI ATM CI --- .../algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 | 2 +- .../atmosphere/background_error_hybrid_gsibec_bump.yaml.j2 | 2 ++ .../model/atmosphere/final_increment_cubed_sphere.yaml.j2 | 2 +- ...twind_ascat_metop-b.yaml.j2 => ascatw.ascat_metop-b.yaml.j2} | 0 .../{satwind_abi_goes-16.yaml.j2 => satwnd.abi_goes-16.yaml.j2} | 0 ...twind_ascat_metop-b.yaml.j2 => ascatw.ascat_metop-b.yaml.j2} | 0 .../{satwind_abi_goes-16.yaml.j2 => satwnd.abi_goes-16.yaml.j2} | 0 7 files changed, 4 insertions(+), 2 deletions(-) rename parm/jcb-gdas/observations/atmosphere-lgetkf/{scatwind_ascat_metop-b.yaml.j2 => ascatw.ascat_metop-b.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere-lgetkf/{satwind_abi_goes-16.yaml.j2 => satwnd.abi_goes-16.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{scatwind_ascat_metop-b.yaml.j2 => ascatw.ascat_metop-b.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{satwind_abi_goes-16.yaml.j2 => satwnd.abi_goes-16.yaml.j2} (100%) diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 index c6fa86557..5bb86a8ba 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 @@ -54,7 +54,7 @@ members: state variables: *bkgvars jedi increment input: filetype: cube sphere history - filename: ./anl/atminc.%yyyy%mm%dd_%hh%MM%ssz.nc4 + filename: ./anl/atminc.%yyyy%mm%dd.%hh%MM%ssz.nc4 provider: ufs fv3 increment output: filetype: auxgrid diff --git a/parm/jcb-gdas/model/atmosphere/background_error_hybrid_gsibec_bump.yaml.j2 b/parm/jcb-gdas/model/atmosphere/background_error_hybrid_gsibec_bump.yaml.j2 index 7802eaac4..440ed74ad 100644 --- a/parm/jcb-gdas/model/atmosphere/background_error_hybrid_gsibec_bump.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/background_error_hybrid_gsibec_bump.yaml.j2 @@ -61,6 +61,8 @@ components: level for 2d variables: last nicas: resolution: 6 + min effective resolution: 0.2 # Note: this is very much not recommended (resolution too low), only + # used because covariance quality is not important for CI tests explicit length-scales: true horizontal length-scale: - groups: diff --git a/parm/jcb-gdas/model/atmosphere/final_increment_cubed_sphere.yaml.j2 b/parm/jcb-gdas/model/atmosphere/final_increment_cubed_sphere.yaml.j2 index a95cef127..247810768 100644 --- a/parm/jcb-gdas/model/atmosphere/final_increment_cubed_sphere.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/final_increment_cubed_sphere.yaml.j2 @@ -1,7 +1,7 @@ output: state component: filetype: cube sphere history - filename: ./anl/atminc.%yyyy%mm%dd_%hh%MM%ssz.nc4 + filename: ./anl/atminc.%yyyy%mm%dd.%hh%MM%ssz.nc4 provider: ufs fields to write: [ugrd,vgrd,tmp,pressfc,spfh,icmr,clwmr,o3mr] geometry: diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/scatwind_ascat_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/ascatw.ascat_metop-b.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere-lgetkf/scatwind_ascat_metop-b.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere-lgetkf/ascatw.ascat_metop-b.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwind_abi_goes-16.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-16.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere-lgetkf/satwind_abi_goes-16.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-16.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/scatwind_ascat_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ascatw.ascat_metop-b.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/scatwind_ascat_metop-b.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/ascatw.ascat_metop-b.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/satwind_abi_goes-16.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/satwind_abi_goes-16.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 From 7a05a090aa03ff6b7d57aad760ed072226ab89e5 Mon Sep 17 00:00:00 2001 From: DavidNew-NOAA Date: Thu, 9 May 2024 13:50:35 +0000 Subject: [PATCH 026/199] Update --- .../algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 | 6 +++--- .../output_ensemble_increments_cubed_sphere.yaml.j2 | 3 +++ 2 files changed, 6 insertions(+), 3 deletions(-) create mode 100644 parm/jcb-gdas/model/atmosphere/output_ensemble_increments_cubed_sphere.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 index e07592e4f..d7b55d07c 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 @@ -1,6 +1,6 @@ variable change: variable change name: Model2GeoVaLs - input variables: &bkgvars [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr,surface_geopotential_height] + input variables: &bkgvars [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr,phis] output variables: &fv3incrvars [ua,va,t,sphum,ice_wat,liq_wat,o3mr,delp,hydrostatic_delz] jedi increment variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] fv3 increment variables: *fv3incrvars @@ -15,7 +15,7 @@ background geometry: npx: {{ npx_ges }} npy: {{ npy_ges }} npz: {{ npz_ges }} - field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_history.yaml + field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_restart.yaml jedi increment geometry: fms initialization: namelist filename: ./fv3jedi/fmsmpp.nml @@ -27,7 +27,7 @@ jedi increment geometry: npx: {{ npx_ges }} npy: {{ npy_ges }} npz: {{ npz_ges }} - field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_history.yaml + field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_restart.yaml fv3 increment geometry: fms initialization: namelist filename: ./fv3jedi/fmsmpp.nml diff --git a/parm/jcb-gdas/model/atmosphere/output_ensemble_increments_cubed_sphere.yaml.j2 b/parm/jcb-gdas/model/atmosphere/output_ensemble_increments_cubed_sphere.yaml.j2 new file mode 100644 index 000000000..c4aa0ea76 --- /dev/null +++ b/parm/jcb-gdas/model/atmosphere/output_ensemble_increments_cubed_sphere.yaml.j2 @@ -0,0 +1,3 @@ +filetype: cube sphere history +provider: ufs +filename: ./anl/mem%{member}%/atminc.%yyyy%mm%dd.%hh%MM%ssz.nc4 \ No newline at end of file From 06582aa3c406042345716336cd95e02eac06b3df Mon Sep 17 00:00:00 2001 From: DavidNew-NOAA Date: Thu, 9 May 2024 18:49:28 +0000 Subject: [PATCH 027/199] Initial commit --- .../atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 index d7b55d07c..eb94c89cb 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 @@ -43,12 +43,16 @@ fv3 increment geometry: members from template: template: background input: - datetime: '{{ atm_background_time_iso }}' - filetype: cube sphere history - provider: ufs datapath: ./bkg/mem%mem% - filename: {{ EPREFIX }}atmf006.nc - state variables: *bkgvars + filetype: fms restart + datetime: '{{ atm_background_time_iso }}' + filename is datetime templated: true + filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" + filename_trcr: "%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc" + filename_sfcd: "%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc" + filename_sfcw: "%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc" + filename_cplr: "%yyyy%mm%dd.%hh%MM%ss.coupler.res" + state variables: *bkgvars jedi increment input: filetype: cube sphere history filename: ./anl/mem%mem%/atminc.%yyyy%mm%dd.%hh%MM%ssz.nc4 From 8fbbaa49f26d66bb49e77ad47231f13de353ce43 Mon Sep 17 00:00:00 2001 From: DavidNew-NOAA <134300700+DavidNew-NOAA@users.noreply.github.com> Date: Fri, 10 May 2024 09:50:20 -0400 Subject: [PATCH 028/199] Update output_ensemble_increments_cubed_sphere.yaml.j2 --- .../atmosphere/output_ensemble_increments_cubed_sphere.yaml.j2 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/parm/jcb-gdas/model/atmosphere/output_ensemble_increments_cubed_sphere.yaml.j2 b/parm/jcb-gdas/model/atmosphere/output_ensemble_increments_cubed_sphere.yaml.j2 index c4aa0ea76..cf655f1c7 100644 --- a/parm/jcb-gdas/model/atmosphere/output_ensemble_increments_cubed_sphere.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/output_ensemble_increments_cubed_sphere.yaml.j2 @@ -1,3 +1,3 @@ filetype: cube sphere history provider: ufs -filename: ./anl/mem%{member}%/atminc.%yyyy%mm%dd.%hh%MM%ssz.nc4 \ No newline at end of file +filename: ./anl/mem%{member}%/atminc.%yyyy%mm%dd.%hh%MM%ssz.nc4 From c70504c3126b09e364ba095298db9442b97a241c Mon Sep 17 00:00:00 2001 From: Brett Hoover Date: Mon, 13 May 2024 10:58:32 -0500 Subject: [PATCH 029/199] Added observations/atmosphere/satwnd.leogeo_multi.yaml.j2 for LEOGEO satwind QC filters --- .../atmosphere/satwnd.leogeo_multi.yaml.j2 | 260 ++++++++++++++++++ 1 file changed, 260 insertions(+) create mode 100644 parm/jcb-gdas/observations/atmosphere/satwnd.leogeo_multi.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.leogeo_multi.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.leogeo_multi.yaml.j2 new file mode 100644 index 000000000..b267c33a4 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.leogeo_multi.yaml.j2 @@ -0,0 +1,260 @@ +- obs space: + name: satwind_leogeo_multi + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + obs operator: + name: VertInterp + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + linear obs operator: + name: VertInterp + + # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI's thinning of AHI/Himawari-8 satwinds + # results in more JEDI satwinds in the diag file than in GSI, but far fewer JEDI satwinds assimilated than + # GSI. JEDI under-counts assimilated winds by roughly 25-40%, relative to GSI, and this under-count is not + # even including the temporal thinning which is applied in GSI but not JEDI (by this filter below). See + # GDASApp Issue #741 for details: https://github.com/NOAA-EMC/GDASApp/issues/741 + #obs pre filters: + #- filter: Gaussian Thinning + # horizontal_mesh: 200 + # vertical_mesh: 10000 + # use_reduced_horizontal_grid: true + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: False + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + + obs post filters: + # Assign the initial observation error, based on height/pressure + # Hard-wiring to prepobs_errtable.global by Type + # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML + + # Type 255 (LEOGEO LWIR) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 255 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000.,105000.,100000.,95000.,90000.,85000.,80000.,75000.,70000.,65000.,60000.,55000.,50000.,45000.,40000.,35000.,30000.,25000.,20000.,15000.,10000.,7500.,5000.,4000.,3000.,2000.,1000.,500.,400.,300.,200.,100.,0.] #Pressure (Pa) + errors: [3.8,3.8,3.8,3.8,3.8,3.8,3.8,3.8,3.9,3.9,4.,4.,4.1,5.,6.,6.3,6.6,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.] + + # sanity-check criteria + # Observation Range Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130. + maxvalue: 130. + action: + name: reject + + # Velocity Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130. + action: + name: reject + + # GSI setupw routine QC + # Reject any ob Type [240–260] when pressure greater than 950 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240-260 + test variables: + - name: MetaData/pressure + maxvalue: 95001. + action: + name: reject + + # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation + # Turn this check off for now. + # Need to check if troposphere pressure was implemented correctly in fv3-jed + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + reference: GeoVaLs/tropopause_pressure + value: MetaData/pressure + minvalue: -5000. # 50 hPa above tropopause level, negative p-diff + action: + name: reject + + # All satwinds must adjust errors based on ObsErrorFactorPressureCheck + # prior to the SPDB check (i.e. the gross-error check). The gross-error + # check uses the adjusted errors for error-bound tightening and rejection, + # so this check has to come first. This check will inflate errors for obs + # that are too close to either the model top or bottom. + # Notes (eliu): GMAO added a required parameter: adjusted_error_name. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + surface_obs: false + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All satwinds subject to a gross-error check that contains significant + # modifiers for satwinds with a negative speed-bias. ALL wind gross-error + # checks are currently being done by the SatWindsSPDBCheck. + # CLEARED + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [ 2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [ 2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don't want to inflate errors for duplication if one + # of the duplicates should be rejected. + # Notes (eliu): ObsErrorFactorDuplicateCheck obsfunction requires PreUseFlag (usage parameter from read_satwnd.f90). + # : Turn off duplicate check for now. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + + # We are extending this to an additional filter that inflates final ob-errors across-the-board by + # 1/0.8 = 1.25. This is caused by the GSI value of nvqc being set to .true. in the global operational + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this nvqc functionality were to be switched off (i.e. if variational qc were to be turned off), + # you would want to remove this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + + # End of Filters From 7cf60c93f5399a09759b8a47be418f2f3d407e85 Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Thu, 16 May 2024 14:48:55 -0400 Subject: [PATCH 030/199] Create fv3jedi_land_ensrecenter.yaml.j2 --- .../land/fv3jedi_land_ensrecenter.yaml.j2 | 63 +++++++++++++++++++ 1 file changed, 63 insertions(+) create mode 100644 parm/jcb-gdas/algorithm/land/fv3jedi_land_ensrecenter.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/land/fv3jedi_land_ensrecenter.yaml.j2 b/parm/jcb-gdas/algorithm/land/fv3jedi_land_ensrecenter.yaml.j2 new file mode 100644 index 000000000..d2ae2962b --- /dev/null +++ b/parm/jcb-gdas/algorithm/land/fv3jedi_land_ensrecenter.yaml.j2 @@ -0,0 +1,63 @@ +geometry: + fms initialization: + namelist filename: ./fv3jedi/fmsmpp.nml + field table filename: ./fv3jedi/field_table + akbk: ./fv3jedi/akbk.nc4 + layout: + - {{ layout_x }} + - {{ layout_y }} + npx: {{ npx_ges }} + npy: {{ npy_ges }} + npz: {{ npz_ges }} + field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_restart.yaml +date: "{{ atm_background_time_iso }}" +variables: [totalSnowDepth] +increment mask: + variable: [slmsk] + maxvalue: 0.99 +deterministic increment: + datetime: "{{ atm_background_time_iso }}" + filetype: fms restart + state variables: [snodl,vtype,slmsk] + datapath: ./anl + filename_core: "%yyyy%mm%dd.%hh%MM%ss.snowinc.fv_core.res.nc" + filename_trcr: "%yyyy%mm%dd.%hh%MM%ss.snowinc.fv_tracer.res.nc" + filename_sfcd: "%yyyy%mm%dd.%hh%MM%ss.snowinc.sfc_data.nc" + filename_sfcw: "%yyyy%mm%dd.%hh%MM%ss.snowinc.fv_srf_wnd.res.nc" + filename_cplr: "%yyyy%mm%dd.%hh%MM%ss.snowinc.coupler.res" +deterministic background: + datetime: "{{ atm_background_time_iso }}" + filetype: fms restart + state variables: [snodl,vtype,slmsk] + datapath: {{ background_path }} + filename is datetime templated: true + filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" + filename_trcr: "%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc" + filename_sfcd: "%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc" + filename_sfcw: "%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc" + filename_cplr: "%yyyy%mm%dd.%hh%MM%ss.coupler.res" +recenter to: deterministic +output increment: + datapath: ./anl + datetime: "{{ atm_background_time_iso }}" + filetype: fms restart + filename_core: inc.fv_core.res.nc + filename_trcr: inc.fv_tracer.res.nc + filename_sfcd: inc.sfc_data.nc + filename_sfcw: inc.fv_srf_wnd.res.nc + filename_cplr: inc.coupler.res +ensemble backgrounds: + pattern: "{{ ensemble_pattern | default("%mem%", true) }}" + number of members: {{ number_ensemble_members }} + zero padding: 3 + template: + datetime: "{{ atm_background_time_iso }}" + filetype: fms restart + state variables: [snodl,vtype,slmsk] + datapath: {{ background_ensemble_path }} + filename is datetime templated: true + filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" + filename_trcr: "%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc" + filename_sfcd: "%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc" + filename_sfcw: "%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc" + filename_cplr: "%yyyy%mm%dd.%hh%MM%ss.coupler.res" From 22ad0602d070db514e17dc0a3e1e6f73e20d1961 Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Fri, 17 May 2024 15:20:39 -0400 Subject: [PATCH 031/199] land to snow --- .../algorithm/{land => snow}/fv3jedi_land_ensrecenter.yaml.j2 | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename parm/jcb-gdas/algorithm/{land => snow}/fv3jedi_land_ensrecenter.yaml.j2 (100%) diff --git a/parm/jcb-gdas/algorithm/land/fv3jedi_land_ensrecenter.yaml.j2 b/parm/jcb-gdas/algorithm/snow/fv3jedi_land_ensrecenter.yaml.j2 similarity index 100% rename from parm/jcb-gdas/algorithm/land/fv3jedi_land_ensrecenter.yaml.j2 rename to parm/jcb-gdas/algorithm/snow/fv3jedi_land_ensrecenter.yaml.j2 From 9dce30ff75e80bfbe1479febe837ea13c8b3643f Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Fri, 17 May 2024 15:38:15 -0400 Subject: [PATCH 032/199] another datetime template --- parm/jcb-gdas/algorithm/snow/fv3jedi_land_ensrecenter.yaml.j2 | 1 + 1 file changed, 1 insertion(+) diff --git a/parm/jcb-gdas/algorithm/snow/fv3jedi_land_ensrecenter.yaml.j2 b/parm/jcb-gdas/algorithm/snow/fv3jedi_land_ensrecenter.yaml.j2 index d2ae2962b..996861169 100644 --- a/parm/jcb-gdas/algorithm/snow/fv3jedi_land_ensrecenter.yaml.j2 +++ b/parm/jcb-gdas/algorithm/snow/fv3jedi_land_ensrecenter.yaml.j2 @@ -20,6 +20,7 @@ deterministic increment: filetype: fms restart state variables: [snodl,vtype,slmsk] datapath: ./anl + filename is datetime templated: true filename_core: "%yyyy%mm%dd.%hh%MM%ss.snowinc.fv_core.res.nc" filename_trcr: "%yyyy%mm%dd.%hh%MM%ss.snowinc.fv_tracer.res.nc" filename_sfcd: "%yyyy%mm%dd.%hh%MM%ss.snowinc.sfc_data.nc" From 90ba52b64cdff069a3d1b057cf54dd71dec2b94c Mon Sep 17 00:00:00 2001 From: Brett Hoover Date: Mon, 20 May 2024 15:32:53 -0500 Subject: [PATCH 033/199] Overhaul satwnd observations/atmosphere/ .yaml.j2 files to update name-conventions: satwind->satwnd, qualityInformationWithoutForecast->qiWithoutForecast --- .../observations/atmosphere/satwind.yaml.j2 | 982 ------------------ .../atmosphere/satwnd.abi_goes-16.yaml.j2 | 4 +- ...-17.yaml.j2 => satwnd.abi_goes-17.yaml.j2} | 4 +- ...d_ahi_h8.yaml.j2 => satwnd.ahi_h8.yaml.j2} | 4 +- ...r_n15.yaml.j2 => satwnd.avhrr_n15.yaml.j2} | 0 ...r_n18.yaml.j2 => satwnd.avhrr_n18.yaml.j2} | 0 ...r_n19.yaml.j2 => satwnd.avhrr_n19.yaml.j2} | 0 .../atmosphere/satwnd.modis_aqua.yaml.j2 | 356 +++++++ .../atmosphere/satwnd.modis_terra.yaml.j2 | 368 +++++++ ..._m11.yaml.j2 => satwnd.seviri_m11.yaml.j2} | 4 +- ...ri_m8.yaml.j2 => satwnd.seviri_m8.yaml.j2} | 4 +- .../atmosphere/satwnd.viirs_n20.yaml.j2 | 289 ++++++ .../atmosphere/satwnd.viirs_npp.yaml.j2 | 289 ++++++ 13 files changed, 1312 insertions(+), 992 deletions(-) delete mode 100644 parm/jcb-gdas/observations/atmosphere/satwind.yaml.j2 rename parm/jcb-gdas/observations/atmosphere/{satwind_abi_goes-17.yaml.j2 => satwnd.abi_goes-17.yaml.j2} (99%) rename parm/jcb-gdas/observations/atmosphere/{satwind_ahi_h8.yaml.j2 => satwnd.ahi_h8.yaml.j2} (99%) rename parm/jcb-gdas/observations/atmosphere/{satwind_avhrr_n15.yaml.j2 => satwnd.avhrr_n15.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{satwind_avhrr_n18.yaml.j2 => satwnd.avhrr_n18.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{satwind_avhrr_n19.yaml.j2 => satwnd.avhrr_n19.yaml.j2} (100%) create mode 100644 parm/jcb-gdas/observations/atmosphere/satwnd.modis_aqua.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/satwnd.modis_terra.yaml.j2 rename parm/jcb-gdas/observations/atmosphere/{satwind_seviri_m11.yaml.j2 => satwnd.seviri_m11.yaml.j2} (99%) rename parm/jcb-gdas/observations/atmosphere/{satwind_seviri_m8.yaml.j2 => satwnd.seviri_m8.yaml.j2} (99%) create mode 100644 parm/jcb-gdas/observations/atmosphere/satwnd.viirs_n20.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/satwnd.viirs_npp.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/satwind.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwind.yaml.j2 deleted file mode 100644 index 3973c3e93..000000000 --- a/parm/jcb-gdas/observations/atmosphere/satwind.yaml.j2 +++ /dev/null @@ -1,982 +0,0 @@ -- - - # Observation Space (I/O) - # ----------------------- - obs space: - name: satwind - obsdatain: - engine: - type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" - obsdataout: - engine: - type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" - io pool: - max pool size: 1 - simulated variables: [windEastward, windNorthward] - - # Observation Operator - # -------------------- - obs operator: - name: VertInterp - - # Observation Pre Filters (QC) - # ---------------------------- - obs pre filters: - # Assign the initial observation error, based on height/pressure - # Hard-wiring to prepobs_errtable.global by Type - # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML - # Type 240 (GOES SWIR): Assigned all dummy values in prepobs_errtable.global - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 240 - minvalue: -135. - maxvalue: 135. - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [110000., 0.] #Pressure (Pa) - errors: [1000000000., 1000000000.] - # Type 241 (Multi Spec. Imager LWIR): Assigned all dummy values - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 241 - minvalue: -135. - maxvalue: 135. - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [110000., 0.] #Pressure (Pa) - errors: [1000000000., 1000000000.] - # Type 242 (Himawari VIS) - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 242 - minvalue: -135. - maxvalue: 135. - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., - 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., - 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., - 500., 400., 300., 200., 100., 0.] #Pressure (Pa) - errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, - 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., - 7., 7., 7.] - # Type 243 (MVIRI/SEVIRI VIS) - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 243 - minvalue: -135. - maxvalue: 135. - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., - 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., - 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., - 500., 400., 300., 200., 100., 0.] #Pressure (Pa) - errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, - 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., - 7., 7., 7.] - # Type 244 (AVHRR LWIR) - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 244 - minvalue: -135. - maxvalue: 135. - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., - 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., - 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., - 500., 400., 300., 200., 100., 0.] #Pressure (Pa) - errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, - 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., - 7., 7., 7.] - # Type 245 (GOES LWIR): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 245 - minvalue: -135. - maxvalue: 135. - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., - 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., - 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., - 500., 400., 300., 200., 100., 0.] #Pressure (Pa) - errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, - 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., - 14., 14., 14., 14., 14., 14.] - # Type 246 (GOES cloud-top WV): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 246 - minvalue: -135. - maxvalue: 135. - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., - 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., - 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., - 500., 400., 300., 200., 100., 0.] #Pressure (Pa) - errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, - 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., - 14., 14., 14., 14., 14., 14.] - # Type 247 (GOES clear-sky WV): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 247 - minvalue: -135. - maxvalue: 135. - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., - 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., - 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., - 500., 400., 300., 200., 100., 0.] #Pressure (Pa) - errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, - 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., - 14., 14., 14., 14., 14., 14.] - # Type 248 (GOES Sounder cloud-top WV): Assigned all dummy values - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 248 - minvalue: -135. - maxvalue: 135. - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [110000., 0.] #Pressure (Pa) - errors: [1000000000., 1000000000.] - # Type 249 (GOES Sounder clear-sky WV): Assigned all dummy values - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 249 - minvalue: -135. - maxvalue: 135. - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [110000., 0.] #Pressure (Pa) - errors: [1000000000., 1000000000.] - # Type 250 (Himawari AHI WV, cloud-top or clear-sky) - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 250 - minvalue: -135. - maxvalue: 135. - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., - 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., - 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., - 500., 400., 300., 200., 100., 0.] #Pressure (Pa) - errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, - 5., 7., 7.3, 7.6, 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., - 8., 8., 8.] - # Type 251 (GOES VIS): Assigned all dummy values - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 251 - minvalue: -135. - maxvalue: 135. - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [110000., 0.] #Pressure (Pa) - errors: [1000000000., 1000000000.] - # Type 252 (Himawari AHI LWIR) - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 252 - minvalue: -135. - maxvalue: 135. - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., - 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., - 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., - 500., 400., 300., 200., 100., 0.] #Pressure (Pa) - errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, - 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., - 7., 7., 7.] - # Type 253 (MVIRI/SEVERI LWIR) - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 253 - minvalue: -135. - maxvalue: 135. - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., - 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., - 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., - 500., 400., 300., 200., 100., 0.] #Pressure (Pa) - errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, - 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., - 7., 7., 7.] - # Type 254 (MVIRI/SEVIRI WV, both cloud-top and clear-sky) - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 254 - minvalue: -135. - maxvalue: 135. - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., - 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., - 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., - 500., 400., 300., 200., 100., 0.] #Pressure (Pa) - errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4.5, 6.1, - 6., 6.5, 7.3, 7.6, 7., 7.5, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., - 7., 7., 7.] - # Type 255 (GOES low-level picture triplet cloud drift): According to prepbufr table this should no longer exist?? - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 255 - minvalue: -135. - maxvalue: 135. - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., - 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., - 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., - 500., 400., 300., 200., 100., 0.] #Pressure (Pa) - errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, - 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., - 7., 7., 7.] - # Type 256 (Multi Spec. Imager WV, both clear-sky and cloud-top): Assigned all dummy values - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 256 - minvalue: -135. - maxvalue: 135. - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [110000., 0.] #Pressure (Pa) - errors: [1000000000., 1000000000.] - # Type 257 (MODIS LWIR) - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 257 - minvalue: -135. - maxvalue: 135. - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., - 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., - 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., - 500., 400., 300., 200., 100., 0.] #Pressure (Pa) - errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, - 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., - 7., 7., 7.] - # Type 258 (MODIS cloud-top WV): Some levels assigned dummy values - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 258 - minvalue: -135. - maxvalue: 135. - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [110000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., - 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., - 500., 400., 300., 200., 100., 0.] #Pressure (Pa) - errors: [1000000000., 1000000000., 4., 4.1, 5., 6., 6.3, 6.6, 7., 7., 7., - 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7.] - # Type 259 (MODIS clear-sky WV): Some levels assigned dummy values - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 259 - minvalue: -135. - maxvalue: 135. - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [110000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., - 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., - 500., 400., 300., 200., 100., 0.] #Pressure (Pa) - errors: [1000000000., 1000000000., 4., 4.1, 5., 6., 6.3, 6.6, 7., 7., 7., - 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7.] - # Type 260 (VIIRS LWIR): All levels assigned dummy values in prepobs_errtable.global, HOWEVER the GSI values appear - # to be a standard profile (borrowed from e.g., Type=244). Using the standard profile here. - # It"s possibly that my prepobs_errtable.global file is out-of-date. - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 260 - minvalue: -135. - maxvalue: 135. - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., - 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., - 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., - 500., 400., 300., 200., 100., 0.] #Pressure (Pa) - errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, - 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., - 7., 7., 7.] - - # Observation Prior Filters (QC) - # ------------------------------ - obs prior filters: - # - # sanity-check criteria - # - # Observation Range Sanity Check - # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - minvalue: -130. - maxvalue: 130. - action: - name: reject - # Velocity Sanity Check - # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - test variables: - - name: ObsFunction/Velocity - maxvalue: 130. - action: - name: reject - # - # preQC (read_satwnd) criteria - # - # EUMETSAT winds: satelliteIdentifer [50–79] (>49, <80) - # Reject obs with satelliteZenithAngle > 68 deg - # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: MetaData/satelliteIdentifier - minvalue: 49. - maxvalue: 80. - test variables: - - name: MetaData/satelliteZenithAngle - maxvalue: 68. - action: - name: reject - # Reject obs with windComputationMethod = 5 (clear-sky WV AMV) - # CLEARED - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: MetaData/satelliteIdentifier - minvalue: 49. - maxvalue: 80. - test variables: - - name: MetaData/windComputationMethod - maxvalue: 4. - action: - name: reject - # Reject obs with qualityInformationWithoutForecast < 85. - # CLEARED - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: MetaData/satelliteIdentifier - minvalue: 49. - maxvalue: 80. - test variables: - - name: MetaData/qualityInformationWithoutForecast - minvalue: 85. - action: - name: reject - # JMA: satelliteIdentifier [100–199] (>99, <200) - # Reject obs with satelliteZenithAngle > 68 deg - # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: MetaData/satelliteIdentifier - minvalue: 99. - maxvalue: 200. - test variables: - - name: MetaData/satelliteZenithAngle - maxvalue: 68. - action: - name: reject - # Reject obs with windComputationMethod = 5 (clear-sky WV AMV) - # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: MetaData/satelliteIdentifier - minvalue: 99. - maxvalue: 200. - test variables: - - name: MetaData/windComputationMethod - maxvalue: 4. - action: - name: reject - # Reject obs with qualityInformationWithoutForecast < 85. - # CLEARED - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: MetaData/satelliteIdentifier - minvalue: 99. - maxvalue: 200. - test variables: - - name: MetaData/qualityInformationWithoutForecast - minvalue: 85. - action: - name: reject - # NESDIS: satelliteIdentifier [250–299] (>249, <300) - # Reject obs with satelliteZenithAngle > 68 deg - # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: MetaData/satelliteIdentifier - minvalue: 249. - maxvalue: 300. - test variables: - - name: MetaData/satelliteZenithAngle - maxvalue: 68. - action: - name: reject - # Reject obs with qualityInformationWithoutForecast < 90. OR > 100. - # CLEARED - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: MetaData/satelliteIdentifier - minvalue: 249. - maxvalue: 300. - test variables: - - name: MetaData/qualityInformationWithoutForecast - minvalue: 90. - maxvalue: 100. - action: - name: reject - # Reject obs with pressure < 15000. - # CLEARED - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: MetaData/satelliteIdentifier - minvalue: 249. - maxvalue: 300. - test variables: - - name: MetaData/pressure - minvalue: 15000. - action: - name: reject - # Reject obs with pressure < 70000. when Type=251 - # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: MetaData/satelliteIdentifier - minvalue: 249. - maxvalue: 300. - - variable: ObsType/windEastward - is_in: 251 - test variables: - - name: MetaData/pressure - minvalue: 70000. - action: - name: reject - # Reject obs with pressure > 30000. when Type=246 - # CLEARED - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: MetaData/satelliteIdentifier - minvalue: 249. - maxvalue: 300. - - variable: ObsType/windEastward - is_in: 246 - test variables: - - name: MetaData/pressure - maxvalue: 30000. - action: - name: reject - # Reject obs with pressure > 85000. when isli=1 (land surface) - # CLEARED - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: MetaData/satelliteIdentifier - minvalue: 249. - maxvalue: 300. - - variable: GeoVaLs/land_type_index_NPOESS - minvalue: 1. - maxvalue: 1. - test variables: - - name: MetaData/pressure - maxvalue: 85000. - action: - name: reject - # Reject obs with pct1 (Coeff. of Var.) outside of 0.04–0.5, Type [240,245,246,251] ONLY - # CLEARED - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: MetaData/satelliteIdentifier - minvalue: 249. - maxvalue: 300. - - variable: ObsType/windEastward - is_in: 240,245,246,251 - test variables: - - name: MetaData/coefficientOfVariation - minvalue: 0.04 - maxvalue: 0.5 - action: - name: reject - # NESDIS obs are also subject to the experr_norm test defined as: - # - # if (10. - 0.1*(expectedError))/(ob_speed)>0.9, or ob_speed<0.1, reject, applies to Types [240,245,246,247,251] - # - # This is not implemented in the YAML file because we do not have the capability to compute - # the norm, lacking the required math operators. Instead, this will likely have to be - # implemented as an ObsFunction like the SatWindsLNVDCheck test. - # - # setupw criteria - # - # Reject any ob Type [240–260] when pressure greater than 950 mb. - # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 240-260 - test variables: - - name: MetaData/pressure - maxvalue: 95001. - action: - name: reject - # GOES IR (245) reject when pressure between 399 and 801 mb. - # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: MetaData/pressure - minvalue: 39901. - maxvalue: 80099. - - variable: ObsType/windEastward - is_in: 245 - action: - name: reject - # JMA IR (252) reject when pressure between 499 and 801 mb. - # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: MetaData/pressure - minvalue: 49901. - maxvalue: 80099. - - variable: ObsType/windEastward - is_in: 252 - action: - name: reject - # EUMETSAT IR (253) reject when pressure between 401 and 801 mb. - # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: MetaData/pressure - minvalue: 40101. - maxvalue: 80099. - - variable: ObsType/windEastward - is_in: 253 - action: - name: reject - # GOES WV (246, 250, 254), reject when pressure greater than 399 mb. - # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 246, 250, 254 - test variables: - - name: MetaData/pressure - maxvalue: 39900. - action: - name: reject - # EUMET (242) and JMA (243) vis, reject when pressure less than 700 mb. - # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 242, 243 - test variables: - - name: MetaData/pressure - minvalue: 70000. - action: - name: reject - # MODIS-Aqua/Terra (257) and (259), reject when pressure less than 249 mb. - # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 257,259 - test variables: - - name: MetaData/pressure - minvalue: 24900. - action: - name: reject - # MODIS-Aqua/Terra (258) and (259), reject when pressure greater than 600 mb. - # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested - # maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 258, 259 - test variables: - - name: MetaData/pressure - maxvalue: 60000. - action: - name: reject - # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. - # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged - - filter: Difference Check - filter variables: - - name: windEastward - - name: windNorthward - reference: GeoVaLs/tropopause_pressure - value: MetaData/pressure - minvalue: -5000. # 50 hPa above tropopause level, negative p-diff - action: - name: reject - # GOES (247) reject any observation with a /=0 surface type (non-water - # surface) within 110 hPa of the surface pressure (as part of the LNVD - # check). - # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested - - filter: Difference Check - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: - name: GeoVaLs/land_type_index_NPOESS - minvalue: 1. - - variable: - name: ObsType/windEastward - is_in: 247 - reference: GeoVaLs/surface_pressure - value: MetaData/pressure - maxvalue: -11000. # within 110 hPa above surface pressure, negative p-diff - action: - name: reject - # AVHRR (244), MODIS (257,258,259), and VIIRS (260) reject any - # observation with a /=0 surface type (non-water surface) within - # 200 hPa of the surface pressure (as part of the LNVD check). - # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged - - filter: Difference Check - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: - name: GeoVaLs/land_type_index_NPOESS - minvalue: 1. - - variable: - name: ObsType/windEastward - is_in: 244, 257-260 - reference: GeoVaLs/surface_pressure - value: MetaData/pressure - maxvalue: -20000. # within 200 hPa above surface pressure, negative p-diff - action: - name: reject - - # Observation Post Filters (QC) - # ----------------------------- - obs post filters: - # Reject GOES (247) when difference of wind direction is more than 50 degrees. - # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 247 - test variables: - - name: ObsFunction/WindDirAngleDiff - maxvalue: 50. - action: - name: reject - # AVHRR (244), MODIS (257,258,259), VIIRS (260), GOES (247) use a LNVD check. - # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 244, 247, 257-260 - test variables: - - name: ObsFunction/SatWindsLNVDCheck - maxvalue: 3. - action: - name: reject - # - # All satwinds subject to a SPDB check (function may be broken?) - # We are ignoring this filter for now, and will come back to - # the issue of how to handle this SPDB check later. For now, - # observations that are GSI-rejected based on this test are - # not being considered when checking acceptance compliance. - #- filter: Bounds Check - # filter variables: - # - name: windEastward - # - name: windNorthward - # test variables: - # - name: ObsFunction/SatWindsSPDBCheck - # options: - # error_min: 1.4 - # error_max: 20.0 - # maxvalue: 1.75 - # action: - # name: reject - # defer to post: true - - # Linear Observation Operator - # --------------------------- - linear obs operator: - name: Identity - - - - # GeoVaLs for Driving Observation Operators (testing mode) - # -------------------------------------------------------- - geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" - - # Passed benchmark for UFO testing - # -------------------------------- - passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 index e7b9c62e5..21c0f49f5 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 @@ -243,7 +243,7 @@ name: reject # GSI read routine QC (part-2) - # Reject obs with qualityInformationWithoutForecast < 90. OR > 100. + # Reject obs with qiWithoutForecast < 90. OR > 100. - filter: Bounds Check filter variables: - name: windEastward @@ -252,7 +252,7 @@ - variable: MetaData/satelliteIdentifier is_in: 250-299 test variables: - - name: MetaData/qualityInformationWithoutForecast + - name: MetaData/qiWithoutForecast minvalue: 90. maxvalue: 100. action: diff --git a/parm/jcb-gdas/observations/atmosphere/satwind_abi_goes-17.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-17.yaml.j2 similarity index 99% rename from parm/jcb-gdas/observations/atmosphere/satwind_abi_goes-17.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-17.yaml.j2 index 3dd8a44f2..bc4b072e4 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwind_abi_goes-17.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-17.yaml.j2 @@ -243,7 +243,7 @@ name: reject # GSI read routine QC (part-2) - # Reject obs with qualityInformationWithoutForecast < 90. OR > 100. + # Reject obs with qiWithoutForecast < 90. OR > 100. - filter: Bounds Check filter variables: - name: windEastward @@ -252,7 +252,7 @@ - variable: MetaData/satelliteIdentifier is_in: 250-299 test variables: - - name: MetaData/qualityInformationWithoutForecast + - name: MetaData/qiWithoutForecast minvalue: 90. maxvalue: 100. action: diff --git a/parm/jcb-gdas/observations/atmosphere/satwind_ahi_h8.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h8.yaml.j2 similarity index 99% rename from parm/jcb-gdas/observations/atmosphere/satwind_ahi_h8.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h8.yaml.j2 index 500f09148..f7569d38f 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwind_ahi_h8.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h8.yaml.j2 @@ -192,13 +192,13 @@ name: reject # GSI read routine QC (part-2) - # Reject obs with qualityInformationWithoutForecast < 85. (also > 100., which is missing data) + # Reject obs with qiWithoutForecast < 85. (also > 100., which is missing data) - filter: Bounds Check filter variables: - name: windEastward - name: windNorthward test variables: - - name: MetaData/qualityInformationWithoutForecast + - name: MetaData/qiWithoutForecast minvalue: 85. maxvalue: 100. action: diff --git a/parm/jcb-gdas/observations/atmosphere/satwind_avhrr_n15.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n15.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/satwind_avhrr_n15.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n15.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/satwind_avhrr_n18.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n18.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/satwind_avhrr_n18.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n18.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/satwind_avhrr_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n19.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/satwind_avhrr_n19.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n19.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.modis_aqua.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.modis_aqua.yaml.j2 new file mode 100644 index 000000000..4ab84f56b --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.modis_aqua.yaml.j2 @@ -0,0 +1,356 @@ +- obs space: + name: satwind_modis_aqua + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + obs operator: + name: VertInterp + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + linear obs operator: + name: VertInterp + + # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI's thinning of AHI/Himawari-8 satwinds + # results in more JEDI satwinds in the diag file than in GSI, but far fewer JEDI satwinds assimilated than + # GSI. JEDI under-counts assimilated winds by roughly 25-40%, relative to GSI, and this under-count is not + # even including the temporal thinning which is applied in GSI but not JEDI (by this filter below). See + # GDASApp Issue #741 for details: https://github.com/NOAA-EMC/GDASApp/issues/741 + #obs pre filters: + #- filter: Gaussian Thinning + # horizontal_mesh: 200 + # vertical_mesh: 10000 + # use_reduced_horizontal_grid: true + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: False + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + + obs post filters: + # Assign the initial observation error, based on height/pressure + # Hard-wiring to prepobs_errtable.global by Type + # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML + + # Type 257 (MODIS LWIR) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 257 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000.,105000.,100000.,95000.,90000.,85000.,80000.,75000.,70000.,65000.,60000.,55000.,50000.,45000.,40000.,35000.,30000.,25000.,20000.,15000.,10000.,7500.,5000.,4000.,3000.,2000.,1000.,500.,400.,300.,200.,100.,0.] #Pressure (Pa) + errors: [3.8,3.8,3.8,3.8,3.8,3.8,3.8,3.8,3.9,3.9,4.,4.,4.1,5.,6.,6.3,6.6,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.] + + # Type 258 (MODIS WVCT): Some levels assigned dummy values + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 258 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000.,60000.,55000.,50000.,45000.,40000.,35000.,30000.,25000.,20000.,15000.,10000.,7500.,5000.,4000.,3000.,2000.,1000.,500.,400.,300.,200.,100.,0.] #Pressure (Pa) + errors: [1000000000.,1000000000.,4.,4.1,5.,6.,6.3,6.6,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.] + + # Type 259 (MODIS WVDL): Some levels assigned dummy values + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 259 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000.,60000.,55000.,50000.,45000.,40000.,35000.,30000.,25000.,20000.,15000.,10000.,7500.,5000.,4000.,3000.,2000.,1000.,500.,400.,300.,200.,100.,0.] #Pressure (Pa) + errors: [1000000000.,1000000000.,4.,4.1,5.,6.,6.3,6.6,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.] + + # sanity-check criteria + # Observation Range Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130. + maxvalue: 130. + action: + name: reject + + # Velocity Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130. + action: + name: reject + + # MODIS-Aqua/Terra (257) and (259), reject when pressure less than 249 mb. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 257,259 + test variables: + - name: MetaData/pressure + minvalue: 24900. + action: + name: reject + + # MODIS-Aqua/Terra (258) and (259), reject when pressure greater than 600 mb. + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + # maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 258, 259 + test variables: + - name: MetaData/pressure + maxvalue: 60000. + action: + name: reject + + # Reject any observation with a /=0 surface type (non-water surface) within + # 200 hPa of the surface pressure (as part of the LNVD check). + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + reference: GeoVaLs/surface_pressure + value: MetaData/pressure + maxvalue: -20000. # within 200 hPa above surface pressure, negative p-diff + action: + name: reject + + # LNVD check + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/SatWindsLNVDCheck + maxvalue: 3. + action: + name: reject + + # GSI setupw routine QC + # Reject any ob Type [240–260] when pressure greater than 950 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240-260 + test variables: + - name: MetaData/pressure + maxvalue: 95001. + action: + name: reject + + # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation + # Turn this check off for now. + # Need to check if troposphere pressure was implemented correctly in fv3-jed + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + reference: GeoVaLs/tropopause_pressure + value: MetaData/pressure + minvalue: -5000. # 50 hPa above tropopause level, negative p-diff + action: + name: reject + + # All satwinds must adjust errors based on ObsErrorFactorPressureCheck + # prior to the SPDB check (i.e. the gross-error check). The gross-error + # check uses the adjusted errors for error-bound tightening and rejection, + # so this check has to come first. This check will inflate errors for obs + # that are too close to either the model top or bottom. + # Notes (eliu): GMAO added a required parameter: adjusted_error_name. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + surface_obs: false + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All satwinds subject to a gross-error check that contains significant + # modifiers for satwinds with a negative speed-bias. ALL wind gross-error + # checks are currently being done by the SatWindsSPDBCheck. + # CLEARED + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [ 2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [ 2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don't want to inflate errors for duplication if one + # of the duplicates should be rejected. + # Notes (eliu): ObsErrorFactorDuplicateCheck obsfunction requires PreUseFlag (usage parameter from read_satwnd.f90). + # : Turn off duplicate check for now. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + + # We are extending this to an additional filter that inflates final ob-errors across-the-board by + # 1/0.8 = 1.25. This is caused by the GSI value of nvqc being set to .true. in the global operational + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this nvqc functionality were to be switched off (i.e. if variational qc were to be turned off), + # you would want to remove this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + + # End of Filters diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.modis_terra.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.modis_terra.yaml.j2 new file mode 100644 index 000000000..811da8211 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.modis_terra.yaml.j2 @@ -0,0 +1,368 @@ +- obs space: + name: satwind_modis_terra + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + obs operator: + name: VertInterp + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + linear obs operator: + name: VertInterp + + # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI's thinning of AHI/Himawari-8 satwinds + # results in more JEDI satwinds in the diag file than in GSI, but far fewer JEDI satwinds assimilated than + # GSI. JEDI under-counts assimilated winds by roughly 25-40%, relative to GSI, and this under-count is not + # even including the temporal thinning which is applied in GSI but not JEDI (by this filter below). See + # GDASApp Issue #741 for details: https://github.com/NOAA-EMC/GDASApp/issues/741 + #obs pre filters: + #- filter: Gaussian Thinning + # horizontal_mesh: 200 + # vertical_mesh: 10000 + # use_reduced_horizontal_grid: true + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: False + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + + obs post filters: + # Assign the initial observation error, based on height/pressure + # Hard-wiring to prepobs_errtable.global by Type + # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML + + # Type 257 (MODIS LWIR) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 257 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000.,105000.,100000.,95000.,90000.,85000.,80000.,75000.,70000.,65000.,60000.,55000.,50000.,45000.,40000.,35000.,30000.,25000.,20000.,15000.,10000.,7500.,5000.,4000.,3000.,2000.,1000.,500.,400.,300.,200.,100.,0.] #Pressure (Pa) + errors: [3.8,3.8,3.8,3.8,3.8,3.8,3.8,3.8,3.9,3.9,4.,4.,4.1,5.,6.,6.3,6.6,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.] + + # Type 258 (MODIS WVCT): Some levels assigned dummy values + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 258 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000.,60000.,55000.,50000.,45000.,40000.,35000.,30000.,25000.,20000.,15000.,10000.,7500.,5000.,4000.,3000.,2000.,1000.,500.,400.,300.,200.,100.,0.] #Pressure (Pa) + errors: [1000000000.,1000000000.,4.,4.1,5.,6.,6.3,6.6,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.] + + # Type 259 (MODIS WVDL): Some levels assigned dummy values + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 259 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000.,60000.,55000.,50000.,45000.,40000.,35000.,30000.,25000.,20000.,15000.,10000.,7500.,5000.,4000.,3000.,2000.,1000.,500.,400.,300.,200.,100.,0.] #Pressure (Pa) + errors: [1000000000.,1000000000.,4.,4.1,5.,6.,6.3,6.6,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.] + + # sanity-check criteria + # Observation Range Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130. + maxvalue: 130. + action: + name: reject + + # Velocity Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130. + action: + name: reject + + # MODIS-TERRA: reject any WVCT (258) or WVDL (259) satwind + # NOTE: This restriction does NOT apply to MODIS-AQUA + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 258,259 + action: + name: reject + + # MODIS-Aqua/Terra (257) and (259), reject when pressure less than 249 mb. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 257,259 + test variables: + - name: MetaData/pressure + minvalue: 24900. + action: + name: reject + + # MODIS-Aqua/Terra (258) and (259), reject when pressure greater than 600 mb. + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + # maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 258, 259 + test variables: + - name: MetaData/pressure + maxvalue: 60000. + action: + name: reject + + # Reject any observation with a /=0 surface type (non-water surface) within + # 200 hPa of the surface pressure (as part of the LNVD check). + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + reference: GeoVaLs/surface_pressure + value: MetaData/pressure + maxvalue: -20000. # within 200 hPa above surface pressure, negative p-diff + action: + name: reject + + # LNVD check + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/SatWindsLNVDCheck + maxvalue: 3. + action: + name: reject + + # GSI setupw routine QC + # Reject any ob Type [240–260] when pressure greater than 950 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240-260 + test variables: + - name: MetaData/pressure + maxvalue: 95001. + action: + name: reject + + # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation + # Turn this check off for now. + # Need to check if troposphere pressure was implemented correctly in fv3-jed + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + reference: GeoVaLs/tropopause_pressure + value: MetaData/pressure + minvalue: -5000. # 50 hPa above tropopause level, negative p-diff + action: + name: reject + + # All satwinds must adjust errors based on ObsErrorFactorPressureCheck + # prior to the SPDB check (i.e. the gross-error check). The gross-error + # check uses the adjusted errors for error-bound tightening and rejection, + # so this check has to come first. This check will inflate errors for obs + # that are too close to either the model top or bottom. + # Notes (eliu): GMAO added a required parameter: adjusted_error_name. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + surface_obs: false + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All satwinds subject to a gross-error check that contains significant + # modifiers for satwinds with a negative speed-bias. ALL wind gross-error + # checks are currently being done by the SatWindsSPDBCheck. + # CLEARED + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [ 2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [ 2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don't want to inflate errors for duplication if one + # of the duplicates should be rejected. + # Notes (eliu): ObsErrorFactorDuplicateCheck obsfunction requires PreUseFlag (usage parameter from read_satwnd.f90). + # : Turn off duplicate check for now. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + + # We are extending this to an additional filter that inflates final ob-errors across-the-board by + # 1/0.8 = 1.25. This is caused by the GSI value of nvqc being set to .true. in the global operational + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this nvqc functionality were to be switched off (i.e. if variational qc were to be turned off), + # you would want to remove this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + + # End of Filters diff --git a/parm/jcb-gdas/observations/atmosphere/satwind_seviri_m11.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m11.yaml.j2 similarity index 99% rename from parm/jcb-gdas/observations/atmosphere/satwind_seviri_m11.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m11.yaml.j2 index 884562104..58e85d6a1 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwind_seviri_m11.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m11.yaml.j2 @@ -219,13 +219,13 @@ name: reject # GSI read routine QC (part-2) - # Reject obs with qualityInformationWithoutForecast < 85. OR > 100. + # Reject obs with qiWithoutForecast < 85. OR > 100. - filter: Bounds Check filter variables: - name: windEastward - name: windNorthward test variables: - - name: MetaData/qualityInformationWithoutForecast + - name: MetaData/qiWithoutForecast minvalue: 85. maxvalue: 100. action: diff --git a/parm/jcb-gdas/observations/atmosphere/satwind_seviri_m8.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m8.yaml.j2 similarity index 99% rename from parm/jcb-gdas/observations/atmosphere/satwind_seviri_m8.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m8.yaml.j2 index 86c0d4259..aac8a659e 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwind_seviri_m8.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m8.yaml.j2 @@ -219,13 +219,13 @@ name: reject # GSI read routine QC (part-2) - # Reject obs with qualityInformationWithoutForecast < 85. OR > 100. + # Reject obs with qiWithoutForecast < 85. OR > 100. - filter: Bounds Check filter variables: - name: windEastward - name: windNorthward test variables: - - name: MetaData/qualityInformationWithoutForecast + - name: MetaData/qiWithoutForecast minvalue: 85. maxvalue: 100. action: diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_n20.yaml.j2 new file mode 100644 index 000000000..033d1e5b3 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_n20.yaml.j2 @@ -0,0 +1,289 @@ +- obs space: + name: satwind_viirs_n20 + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + obs operator: + name: VertInterp + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + linear obs operator: + name: VertInterp + + # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI's thinning of AHI/Himawari-8 satwinds + # results in more JEDI satwinds in the diag file than in GSI, but far fewer JEDI satwinds assimilated than + # GSI. JEDI under-counts assimilated winds by roughly 25-40%, relative to GSI, and this under-count is not + # even including the temporal thinning which is applied in GSI but not JEDI (by this filter below). See + # GDASApp Issue #741 for details: https://github.com/NOAA-EMC/GDASApp/issues/741 + #obs pre filters: + #- filter: Gaussian Thinning + # horizontal_mesh: 200 + # vertical_mesh: 10000 + # use_reduced_horizontal_grid: true + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: False + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + + obs post filters: + # Assign the initial observation error, based on height/pressure + # Hard-wiring to prepobs_errtable.global by Type + # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML + + # Type 260 (VIIRS LWIR) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 260 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000.,105000.,100000.,95000.,90000.,85000.,80000.,75000.,70000.,65000.,60000.,55000.,50000.,45000.,40000.,35000.,30000.,25000.,20000.,15000.,10000.,7500.,5000.,4000.,3000.,2000.,1000.,500.,400.,300.,200.,100.,0.] + errors: [3.8,3.8,3.8,3.8,3.8,3.8,3.8,3.8,3.9,3.9,4.,4.,4.1,5.,6.,6.3,6.6,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.] + + # sanity-check criteria + # Observation Range Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130. + maxvalue: 130. + action: + name: reject + + # Velocity Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130. + action: + name: reject + + # Reject any observation with a /=0 surface type (non-water surface) within + # 200 hPa of the surface pressure (as part of the LNVD check). + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + reference: GeoVaLs/surface_pressure + value: MetaData/pressure + maxvalue: -20000. # within 200 hPa above surface pressure, negative p-diff + action: + name: reject + + # LNVD check + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/SatWindsLNVDCheck + maxvalue: 3. + action: + name: reject + + # GSI setupw routine QC + # Reject any ob Type [240–260] when pressure greater than 950 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240-260 + test variables: + - name: MetaData/pressure + maxvalue: 95001. + action: + name: reject + + # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation + # Turn this check off for now. + # Need to check if troposphere pressure was implemented correctly in fv3-jed + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + reference: GeoVaLs/tropopause_pressure + value: MetaData/pressure + minvalue: -5000. # 50 hPa above tropopause level, negative p-diff + action: + name: reject + + # All satwinds must adjust errors based on ObsErrorFactorPressureCheck + # prior to the SPDB check (i.e. the gross-error check). The gross-error + # check uses the adjusted errors for error-bound tightening and rejection, + # so this check has to come first. This check will inflate errors for obs + # that are too close to either the model top or bottom. + # Notes (eliu): GMAO added a required parameter: adjusted_error_name. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + surface_obs: false + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All satwinds subject to a gross-error check that contains significant + # modifiers for satwinds with a negative speed-bias. ALL wind gross-error + # checks are currently being done by the SatWindsSPDBCheck. + # CLEARED + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [ 2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [ 2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don't want to inflate errors for duplication if one + # of the duplicates should be rejected. + # Notes (eliu): ObsErrorFactorDuplicateCheck obsfunction requires PreUseFlag (usage parameter from read_satwnd.f90). + # : Turn off duplicate check for now. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + + # We are extending this to an additional filter that inflates final ob-errors across-the-board by + # 1/0.8 = 1.25. This is caused by the GSI value of nvqc being set to .true. in the global operational + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this nvqc functionality were to be switched off (i.e. if variational qc were to be turned off), + # you would want to remove this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + + # End of Filters diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_npp.yaml.j2 new file mode 100644 index 000000000..c8148fbbb --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_npp.yaml.j2 @@ -0,0 +1,289 @@ +- obs space: + name: satwind_viirs_npp + obsdatain: + engine: + type: H5File + obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + obs operator: + name: VertInterp + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + linear obs operator: + name: VertInterp + + # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI's thinning of AHI/Himawari-8 satwinds + # results in more JEDI satwinds in the diag file than in GSI, but far fewer JEDI satwinds assimilated than + # GSI. JEDI under-counts assimilated winds by roughly 25-40%, relative to GSI, and this under-count is not + # even including the temporal thinning which is applied in GSI but not JEDI (by this filter below). See + # GDASApp Issue #741 for details: https://github.com/NOAA-EMC/GDASApp/issues/741 + #obs pre filters: + #- filter: Gaussian Thinning + # horizontal_mesh: 200 + # vertical_mesh: 10000 + # use_reduced_horizontal_grid: true + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: False + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + + obs post filters: + # Assign the initial observation error, based on height/pressure + # Hard-wiring to prepobs_errtable.global by Type + # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML + + # Type 260 (VIIRS LWIR) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 260 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000.,105000.,100000.,95000.,90000.,85000.,80000.,75000.,70000.,65000.,60000.,55000.,50000.,45000.,40000.,35000.,30000.,25000.,20000.,15000.,10000.,7500.,5000.,4000.,3000.,2000.,1000.,500.,400.,300.,200.,100.,0.] + errors: [3.8,3.8,3.8,3.8,3.8,3.8,3.8,3.8,3.9,3.9,4.,4.,4.1,5.,6.,6.3,6.6,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.] + + # sanity-check criteria + # Observation Range Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130. + maxvalue: 130. + action: + name: reject + + # Velocity Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130. + action: + name: reject + + # Reject any observation with a /=0 surface type (non-water surface) within + # 200 hPa of the surface pressure (as part of the LNVD check). + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + reference: GeoVaLs/surface_pressure + value: MetaData/pressure + maxvalue: -20000. # within 200 hPa above surface pressure, negative p-diff + action: + name: reject + + # LNVD check + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/SatWindsLNVDCheck + maxvalue: 3. + action: + name: reject + + # GSI setupw routine QC + # Reject any ob Type [240–260] when pressure greater than 950 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240-260 + test variables: + - name: MetaData/pressure + maxvalue: 95001. + action: + name: reject + + # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation + # Turn this check off for now. + # Need to check if troposphere pressure was implemented correctly in fv3-jed + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + reference: GeoVaLs/tropopause_pressure + value: MetaData/pressure + minvalue: -5000. # 50 hPa above tropopause level, negative p-diff + action: + name: reject + + # All satwinds must adjust errors based on ObsErrorFactorPressureCheck + # prior to the SPDB check (i.e. the gross-error check). The gross-error + # check uses the adjusted errors for error-bound tightening and rejection, + # so this check has to come first. This check will inflate errors for obs + # that are too close to either the model top or bottom. + # Notes (eliu): GMAO added a required parameter: adjusted_error_name. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + surface_obs: false + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All satwinds subject to a gross-error check that contains significant + # modifiers for satwinds with a negative speed-bias. ALL wind gross-error + # checks are currently being done by the SatWindsSPDBCheck. + # CLEARED + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [ 2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [ 2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don't want to inflate errors for duplication if one + # of the duplicates should be rejected. + # Notes (eliu): ObsErrorFactorDuplicateCheck obsfunction requires PreUseFlag (usage parameter from read_satwnd.f90). + # : Turn off duplicate check for now. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + + # We are extending this to an additional filter that inflates final ob-errors across-the-board by + # 1/0.8 = 1.25. This is caused by the GSI value of nvqc being set to .true. in the global operational + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this nvqc functionality were to be switched off (i.e. if variational qc were to be turned off), + # you would want to remove this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + + # End of Filters From 2c424d969b75dd7cec8101e87f7ce8e6583f6f8d Mon Sep 17 00:00:00 2001 From: danholdaway Date: Thu, 23 May 2024 16:40:02 -0400 Subject: [PATCH 034/199] Add action for running JCB tests --- .../.github/run_jcb_basic_testing.yaml | 61 +++++++++++++++++++ 1 file changed, 61 insertions(+) create mode 100644 parm/jcb-gdas/.github/run_jcb_basic_testing.yaml diff --git a/parm/jcb-gdas/.github/run_jcb_basic_testing.yaml b/parm/jcb-gdas/.github/run_jcb_basic_testing.yaml new file mode 100644 index 000000000..de7383495 --- /dev/null +++ b/parm/jcb-gdas/.github/run_jcb_basic_testing.yaml @@ -0,0 +1,61 @@ +name: Run JCB basic testing with jcb-gdas changes + +on: + push: + branches: + - develop + pull_request: + types: + - opened + - synchronize + - reopened + +jobs: + test: + runs-on: ubuntu-latest + + env: + JCB_REPO: https://github.com/NOAA-EMC/jcb.git + JCB_DEVELOP_BRANCH: develop + + steps: + - name: Checkout submodule repository + uses: actions/checkout@v3 + with: + submodules: true + + - name: Set up Python + uses: actions/setup-python@v4 + with: + python-version: '3.x' # specify your python version + + - name: Determine branch to use + id: determine-branch + run: | + # Determine if the branch exists in the jcb repo + BRANCH_NAME=${{ github.ref_name }} + git ls-remote --exit-code --heads $JCB_REPO $BRANCH_NAME + if [ $? -eq 0 ]; then + echo "Branch $BRANCH_NAME exists in jcb repo." + echo "BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_ENV + else + echo "Branch $BRANCH_NAME does not exist in jcb repo. Using develop branch." + echo "BRANCH_NAME=$JCB_DEVELOP_BRANCH" >> $GITHUB_ENV + fi + + - name: Clone jcb repository + run: | + git clone --branch ${{ env.BRANCH_NAME }} $JCB_REPO jcb_repo + cd jcb_repo/src/jcb/configuration/apps/gdas + git checkout ${{ env.BRANCH_NAME }} + cd - + + - name: Install dependencies + run: | + cd jcb_repo + pip install .[testing] + + - name: Run unit tests + run: | + cd $GITHUB_WORKSPACE + pytest -v From 42229a754dd6b25c172dec9716b31d2a9b8d5562 Mon Sep 17 00:00:00 2001 From: danholdaway Date: Thu, 23 May 2024 16:40:51 -0400 Subject: [PATCH 035/199] move --- parm/jcb-gdas/.github/{ => workflows}/run_jcb_basic_testing.yaml | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename parm/jcb-gdas/.github/{ => workflows}/run_jcb_basic_testing.yaml (100%) diff --git a/parm/jcb-gdas/.github/run_jcb_basic_testing.yaml b/parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml similarity index 100% rename from parm/jcb-gdas/.github/run_jcb_basic_testing.yaml rename to parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml From 21e40ff98f3b7277d4e26795c0e5e7206e1c2d10 Mon Sep 17 00:00:00 2001 From: Dan Holdaway <27729500+danholdaway@users.noreply.github.com> Date: Fri, 24 May 2024 10:57:38 -0400 Subject: [PATCH 036/199] Run the JCB client integration tests when making PRs in the client (#11) Co-authored-by: danholdaway --- .../workflows/run_jcb_basic_testing.yaml | 52 ++++++++++++------- 1 file changed, 34 insertions(+), 18 deletions(-) diff --git a/parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml b/parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml index de7383495..283ba9a5d 100644 --- a/parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml +++ b/parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml @@ -1,4 +1,4 @@ -name: Run JCB basic testing with jcb-gdas changes +name: Run JCB client testing with client changes on: push: @@ -16,10 +16,9 @@ jobs: env: JCB_REPO: https://github.com/NOAA-EMC/jcb.git - JCB_DEVELOP_BRANCH: develop steps: - - name: Checkout submodule repository + - name: Checkout application client repository uses: actions/checkout@v3 with: submodules: true @@ -27,35 +26,52 @@ jobs: - name: Set up Python uses: actions/setup-python@v4 with: - python-version: '3.x' # specify your python version + python-version: '3.x' - - name: Determine branch to use - id: determine-branch + - name: Add repo url to the environment run: | - # Determine if the branch exists in the jcb repo - BRANCH_NAME=${{ github.ref_name }} - git ls-remote --exit-code --heads $JCB_REPO $BRANCH_NAME - if [ $? -eq 0 ]; then + JCB_APP_REPO="https://github.com/${{ github.repository }}" + echo "JCB_APP_REPO=${JCB_APP_REPO}" >> $GITHUB_ENV + + - name: Determine the name of the client branch + run: | + if [ "${{ github.event_name }}" == "pull_request" ]; then + JCB_APP_BRANCH=${{ github.head_ref }} + else + BRANCH_REF=${{ github.ref }} + JCB_APP_BRANCH=${BRANCH_REF#refs/heads/} + fi + echo "JCB_APP_BRANCH=$JCB_APP_BRANCH" >> $GITHUB_ENV + + - name: Determine branch to use for main jcb repo + run: | + BRANCH_NAME=${{ env.JCB_APP_BRANCH }} + if git ls-remote --heads $JCB_REPO $BRANCH_NAME | grep -q "refs/heads/$BRANCH_NAME"; then echo "Branch $BRANCH_NAME exists in jcb repo." - echo "BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_ENV + echo "JCB_BRANCH=$BRANCH_NAME" >> $GITHUB_ENV else echo "Branch $BRANCH_NAME does not exist in jcb repo. Using develop branch." - echo "BRANCH_NAME=$JCB_DEVELOP_BRANCH" >> $GITHUB_ENV + echo "JCB_BRANCH=develop" >> $GITHUB_ENV fi - name: Clone jcb repository run: | - git clone --branch ${{ env.BRANCH_NAME }} $JCB_REPO jcb_repo - cd jcb_repo/src/jcb/configuration/apps/gdas - git checkout ${{ env.BRANCH_NAME }} - cd - + mkdir -p empty_hooks + git config --global core.hooksPath empty_hooks + git clone --branch ${{ env.JCB_BRANCH }} --recursive $JCB_REPO jcb_repo + + - name: Switch the application branch to match the action + run: | + cd jcb_repo + cd $(python get_submod_path.py $JCB_APP_REPO) + git checkout ${{ env.JCB_APP_BRANCH }} - name: Install dependencies run: | cd jcb_repo pip install .[testing] - - name: Run unit tests + - name: Run the JCB client integration tests run: | - cd $GITHUB_WORKSPACE + cd jcb_repo/test/client_integration pytest -v From 5c007a211426303db9eb4fd04259dcee0d4690b3 Mon Sep 17 00:00:00 2001 From: danholdaway Date: Fri, 24 May 2024 11:19:47 -0400 Subject: [PATCH 037/199] give the action a more informative name --- parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml b/parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml index 283ba9a5d..523f976a5 100644 --- a/parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml +++ b/parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml @@ -11,8 +11,11 @@ on: - reopened jobs: - test: + + jcb_integration_tests: + runs-on: ubuntu-latest + name: JCB Client Integration Tests env: JCB_REPO: https://github.com/NOAA-EMC/jcb.git From 15529f223c7bd8576571af57dba999bbbc187567 Mon Sep 17 00:00:00 2001 From: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com> Date: Fri, 24 May 2024 13:47:20 -0400 Subject: [PATCH 038/199] Update atmosphere-lgetkf satwnd yamls (#13) Co-authored-by: Dan Holdaway <27729500+danholdaway@users.noreply.github.com> --- .../observations/atmosphere-lgetkf/satwnd.abi_goes-16.yaml.j2 | 4 ++-- .../{satwind_ahi_h8.yaml.j2 => satwnd.ahi_h8.yaml.j2} | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) rename parm/jcb-gdas/observations/atmosphere-lgetkf/{satwind_ahi_h8.yaml.j2 => satwnd.ahi_h8.yaml.j2} (99%) diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-16.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-16.yaml.j2 index 2c276038e..17f601c3c 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-16.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-16.yaml.j2 @@ -246,7 +246,7 @@ name: reject # GSI read routine QC (part-2) - # Reject obs with qualityInformationWithoutForecast < 90. OR > 100. + # Reject obs with qiWithoutForecast < 90. OR > 100. - filter: Bounds Check filter variables: - name: windEastward @@ -255,7 +255,7 @@ - variable: MetaData/satelliteIdentifier is_in: 250-299 test variables: - - name: MetaData/qualityInformationWithoutForecast + - name: MetaData/qiWithoutForecast minvalue: 90. maxvalue: 100. action: diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwind_ahi_h8.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.ahi_h8.yaml.j2 similarity index 99% rename from parm/jcb-gdas/observations/atmosphere-lgetkf/satwind_ahi_h8.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.ahi_h8.yaml.j2 index f4ce28c48..7664fcc57 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwind_ahi_h8.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.ahi_h8.yaml.j2 @@ -195,13 +195,13 @@ name: reject # GSI read routine QC (part-2) - # Reject obs with qualityInformationWithoutForecast < 85. (also > 100., which is missing data) + # Reject obs with qiWithoutForecast < 85. (also > 100., which is missing data) - filter: Bounds Check filter variables: - name: windEastward - name: windNorthward test variables: - - name: MetaData/qualityInformationWithoutForecast + - name: MetaData/qiWithoutForecast minvalue: 85. maxvalue: 100. action: From a8aee98651569c9f9a82027ff59309bd8a27874a Mon Sep 17 00:00:00 2001 From: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com> Date: Wed, 5 Jun 2024 11:34:56 -0400 Subject: [PATCH 039/199] Yaml updates for use with updated GDASApp submodules (#8) GDASApp PR [#1123](https://github.com/NOAA-EMC/GDASApp/pull/1123) updates the hashes for select JEDI repositories. The updated hashes require yaml updates. - add `static` keyword to `gsi covariance` - update names used for radiance bias correction terms Resolves #6 Co-authored-by: Dan Holdaway <27729500+danholdaway@users.noreply.github.com> --- .../background_error_hybrid_gsibec_bump.yaml.j2 | 2 +- .../background_error_static_gsibec.yaml.j2 | 2 +- .../atmosphere-lgetkf/amsua_n19.yaml.j2 | 14 +++++++------- .../observations/atmosphere/amsua_n19.yaml.j2 | 14 +++++++------- 4 files changed, 16 insertions(+), 16 deletions(-) diff --git a/parm/jcb-gdas/model/atmosphere/background_error_hybrid_gsibec_bump.yaml.j2 b/parm/jcb-gdas/model/atmosphere/background_error_hybrid_gsibec_bump.yaml.j2 index 440ed74ad..ef8fe88b6 100644 --- a/parm/jcb-gdas/model/atmosphere/background_error_hybrid_gsibec_bump.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/background_error_hybrid_gsibec_bump.yaml.j2 @@ -3,7 +3,7 @@ components: - covariance: covariance model: SABER saber central block: - saber block name: gsi covariance + saber block name: gsi static covariance read: gsi akbk: &akbk {{fv3jedi_files_path}}/akbk.nc4 gsi error covariance file: &gsiberr {{gsibec_path}}/gsi-coeffs-gfs-global.nc4 diff --git a/parm/jcb-gdas/model/atmosphere/background_error_static_gsibec.yaml.j2 b/parm/jcb-gdas/model/atmosphere/background_error_static_gsibec.yaml.j2 index 86a863302..e53b8fd9d 100644 --- a/parm/jcb-gdas/model/atmosphere/background_error_static_gsibec.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/background_error_static_gsibec.yaml.j2 @@ -1,7 +1,7 @@ covariance model: SABER full inverse: true saber central block: - saber block name: gsi covariance + saber block name: gsi static covariance read: gsi akbk: &akbk {{fv3jedi_files_path}}/akbk.nc4 gsi error covariance file: &gsiberr {{gsibec_path}}/gsi-coeffs-gfs-global.nc4 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 index 94fd7a83c..eff404a12 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 @@ -45,19 +45,19 @@ variational bc: predictors: - name: constant - - name: lapse_rate + - name: lapseRate order: 2 tlapse: &{{observation_from_jcb}}_tlapse "{{atm_obsbiasin_path}}/{{atm_obstlapsein_prefix}}{{observation_from_jcb}}{{atm_obstlapsein_suffix}}" - - name: lapse_rate + - name: lapseRate tlapse: *{{observation_from_jcb}}_tlapse - - name: emissivity - - name: scan_angle + - name: emissivityJacobian + - name: sensorScanAngle order: 4 - - name: scan_angle + - name: sensorScanAngle order: 3 - - name: scan_angle + - name: sensorScanAngle order: 2 - - name: scan_angle + - name: sensorScanAngle covariance: minimal required obs number: 20 variance range: [1.0e-6, 10.0] diff --git a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 index 244a1cd58..d8dd23f28 100644 --- a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 @@ -37,19 +37,19 @@ variational bc: predictors: - name: constant - - name: lapse_rate + - name: lapseRate order: 2 tlapse: &{{observation_from_jcb}}_tlapse "{{atm_obsbiasin_path}}/{{atm_obstlapsein_prefix}}{{observation_from_jcb}}{{atm_obstlapsein_suffix}}" - - name: lapse_rate + - name: lapseRate tlapse: *{{observation_from_jcb}}_tlapse - - name: emissivity - - name: scan_angle + - name: emissivityJacobian + - name: sensorScanAngle order: 4 - - name: scan_angle + - name: sensorScanAngle order: 3 - - name: scan_angle + - name: sensorScanAngle order: 2 - - name: scan_angle + - name: sensorScanAngle covariance: minimal required obs number: 20 variance range: [1.0e-6, 10.0] From 5dd2540614727ceca7a63b343ca61b364011a47e Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Fri, 14 Jun 2024 16:59:35 +0000 Subject: [PATCH 040/199] Changes to the JCB snow ensemble recenter template (#18) Change to JCB templates to not look for snow obs in JCB (they're not there yet, will change this at a later date) --- .../snow/fv3jedi_land_ensrecenter.yaml.j2 | 44 ++++++++++--------- 1 file changed, 24 insertions(+), 20 deletions(-) diff --git a/parm/jcb-gdas/algorithm/snow/fv3jedi_land_ensrecenter.yaml.j2 b/parm/jcb-gdas/algorithm/snow/fv3jedi_land_ensrecenter.yaml.j2 index 996861169..f7236a8c0 100644 --- a/parm/jcb-gdas/algorithm/snow/fv3jedi_land_ensrecenter.yaml.j2 +++ b/parm/jcb-gdas/algorithm/snow/fv3jedi_land_ensrecenter.yaml.j2 @@ -10,55 +10,59 @@ geometry: npy: {{ npy_ges }} npz: {{ npz_ges }} field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_restart.yaml -date: "{{ atm_background_time_iso }}" +date: "{{ snow_background_time_iso }}" variables: [totalSnowDepth] increment mask: variable: [slmsk] - maxvalue: 0.99 + maxvalue: 0.99 deterministic increment: - datetime: "{{ atm_background_time_iso }}" + datetime: "{{ snow_increment_time_iso }}" filetype: fms restart state variables: [snodl,vtype,slmsk] - datapath: ./anl + datapath: ./inc/det_ensres filename is datetime templated: true - filename_core: "%yyyy%mm%dd.%hh%MM%ss.snowinc.fv_core.res.nc" - filename_trcr: "%yyyy%mm%dd.%hh%MM%ss.snowinc.fv_tracer.res.nc" - filename_sfcd: "%yyyy%mm%dd.%hh%MM%ss.snowinc.sfc_data.nc" - filename_sfcw: "%yyyy%mm%dd.%hh%MM%ss.snowinc.fv_srf_wnd.res.nc" - filename_cplr: "%yyyy%mm%dd.%hh%MM%ss.snowinc.coupler.res" + filename_core: "snowinc.%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" + filename_trcr: "snowinc.%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc" + filename_sfcd: "snowinc.%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc" + filename_sfcw: "snowinc.%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc" + filename_cplr: "snowinc.%yyyy%mm%dd.%hh%MM%ss.coupler.res" + skip coupler file: true deterministic background: - datetime: "{{ atm_background_time_iso }}" + datetime: "{{ snow_background_time_iso }}" filetype: fms restart state variables: [snodl,vtype,slmsk] - datapath: {{ background_path }} + datapath: ./bkg/det_ensres filename is datetime templated: true filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" filename_trcr: "%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc" filename_sfcd: "%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc" filename_sfcw: "%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc" filename_cplr: "%yyyy%mm%dd.%hh%MM%ss.coupler.res" + skip coupler file: true recenter to: deterministic output increment: - datapath: ./anl - datetime: "{{ atm_background_time_iso }}" + datapath: ./inc/ensmean + datetime: "{{ snow_increment_time_iso }}" filetype: fms restart - filename_core: inc.fv_core.res.nc - filename_trcr: inc.fv_tracer.res.nc - filename_sfcd: inc.sfc_data.nc - filename_sfcw: inc.fv_srf_wnd.res.nc - filename_cplr: inc.coupler.res + prefix: snowinc + filename_core: "{{ snow_increment_time_fv3 }}.fv_core.res.nc" + filename_trcr: "{{ snow_increment_time_fv3 }}.fv_tracer.res.nc" + filename_sfcd: "{{ snow_increment_time_fv3 }}.sfc_data.nc" + filename_sfcw: "{{ snow_increment_time_fv3 }}.fv_srf_wnd.res.nc" + filename_cplr: "{{ snow_increment_time_fv3 }}.coupler.res" ensemble backgrounds: pattern: "{{ ensemble_pattern | default("%mem%", true) }}" number of members: {{ number_ensemble_members }} zero padding: 3 template: - datetime: "{{ atm_background_time_iso }}" + datetime: "{{ snow_background_time_iso }}" filetype: fms restart state variables: [snodl,vtype,slmsk] - datapath: {{ background_ensemble_path }} + datapath: ./bkg/mem%mem% filename is datetime templated: true filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" filename_trcr: "%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc" filename_sfcd: "%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc" filename_sfcw: "%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc" filename_cplr: "%yyyy%mm%dd.%hh%MM%ss.coupler.res" + skip coupler file: true From 8139f72b99f0ecc652934898ddd49b1ce0950a45 Mon Sep 17 00:00:00 2001 From: Dan Holdaway <27729500+danholdaway@users.noreply.github.com> Date: Tue, 18 Jun 2024 14:06:25 -0400 Subject: [PATCH 041/199] Adjust testing for new path checker file (#16) --- .../workflows/run_jcb_basic_testing.yaml | 23 +++++++++++-------- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml b/parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml index 523f976a5..97714eba5 100644 --- a/parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml +++ b/parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml @@ -21,11 +21,6 @@ jobs: JCB_REPO: https://github.com/NOAA-EMC/jcb.git steps: - - name: Checkout application client repository - uses: actions/checkout@v3 - with: - submodules: true - - name: Set up Python uses: actions/setup-python@v4 with: @@ -33,7 +28,7 @@ jobs: - name: Add repo url to the environment run: | - JCB_APP_REPO="https://github.com/${{ github.repository }}" + JCB_APP_REPO="${{ github.repository }}" echo "JCB_APP_REPO=${JCB_APP_REPO}" >> $GITHUB_ENV - name: Determine the name of the client branch @@ -63,11 +58,21 @@ jobs: git config --global core.hooksPath empty_hooks git clone --branch ${{ env.JCB_BRANCH }} --recursive $JCB_REPO jcb_repo - - name: Switch the application branch to match the action + - name: Clone the clients run: | cd jcb_repo - cd $(python get_submod_path.py $JCB_APP_REPO) - git checkout ${{ env.JCB_APP_BRANCH }} + pip install pyyaml + ./jcb_client_init.py + + # If all the branches are found then at this point everything should be good since + # jcb_client_init will check out the correct branches. If not then the app branch needs to + # be checked out explicitly. + if [ "${{ env.JCB_BRANCH }}" != "${{ env.JCB_APP_BRANCH }}" ]; then + echo "Checking out branch ${{ env.JCB_APP_BRANCH }} for ${{ env.JCB_APP_REPO }}" + app_path=$(python jcb_client_path.py $JCB_APP_REPO) + cd $app_path + git checkout ${{ env.JCB_APP_BRANCH }} + fi - name: Install dependencies run: | From 3835807a8e229049aef44f1fd21f60bed3584712 Mon Sep 17 00:00:00 2001 From: Dan Holdaway <27729500+danholdaway@users.noreply.github.com> Date: Thu, 20 Jun 2024 13:13:57 -0400 Subject: [PATCH 042/199] Companion PR to NOAA-EMC/jcb/pull/7 (#15) - Rename the atmosphere YAML files to be `atmosphere_.yaml.j2` - Change all the keys associated with the atmosphere to be `{{atmosphere_}}` - Add the marine YAML files. --------- Co-authored-by: danholdaway Co-authored-by: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com> --- .../atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 | 36 +- .../fv3jedi_fv3inc_variational.yaml.j2 | 32 +- .../algorithm/marine/socaincr2mom6.yaml | 28 + ...geometry.yaml.j2 => aero_geometry.yaml.j2} | 0 .../atmosphere/3dvar_outer_loop_1.yaml.j2 | 17 - .../atmosphere/3dvar_outer_loop_2.yaml.j2 | 17 - .../atmosphere_3dvar_outer_loop_1.yaml.j2 | 17 + .../atmosphere_3dvar_outer_loop_2.yaml.j2 | 17 + ....yaml.j2 => atmosphere_background.yaml.j2} | 4 +- ...=> atmosphere_background_ensemble.yaml.j2} | 10 +- ...background_error_hybrid_bump_bump.yaml.j2} | 52 +- ...ckground_error_hybrid_gsibec_bump.yaml.j2} | 42 +- ...here_background_error_static_bump.yaml.j2} | 32 +- ...ere_background_error_static_gsibec.yaml.j2 | 25 + ...e_background_error_static_identity.yaml.j2 | 2 + ...phere_final_increment_cubed_sphere.yaml.j2 | 18 + ...tmosphere_final_increment_gaussian.yaml.j2 | 17 + .../atmosphere_geometry_background.yaml.j2 | 11 + ...aml.j2 => atmosphere_model_pseudo.yaml.j2} | 4 +- ..._ensemble_increments_cubed_sphere.yaml.j2} | 0 ...utput_ensemble_increments_gaussian.yaml.j2 | 3 + ...mosphere_posterior_output_gaussian.yaml.j2 | 3 + .../background_error_static_gsibec.yaml.j2 | 25 - .../background_error_static_identity.yaml.j2 | 2 - .../final_increment_cubed_sphere.yaml.j2 | 18 - .../final_increment_gaussian.yaml.j2 | 17 - .../atmosphere/geometry_background.yaml.j2 | 11 - ...utput_ensemble_increments_gaussian.yaml.j2 | 3 - .../posterior_output_gaussian.yaml.j2 | 3 - .../marine/marine_3dfgat_outer_loop_1.yaml.j2 | 7 + .../marine/marine_3dvar_outer_loop_1.yaml.j2 | 7 + .../model/marine/marine_background.yaml.j2 | 6 + ...d_error_hybrid_diffusion_diffusion.yaml.j2 | 92 +++ ..._background_error_static_diffusion.yaml.j2 | 42 ++ .../marine/marine_final_increment.yaml.j2 | 9 + .../marine/marine_geometry_background.yaml.j2 | 2 + .../model/marine/marine_model_pseudo.yaml.j2 | 10 + .../model/marine/marine_output.yaml.j2 | 4 + parm/jcb-gdas/model/snow/geometry.yaml.j2 | 0 .../snow_geometry.yaml.j2} | 0 .../atmosphere-lgetkf/amsua_n19.yaml.j2 | 16 +- .../ascatw.ascat_metop-b.yaml.j2 | 8 +- .../atmosphere-lgetkf/conv_ps.yaml.j2 | 22 +- .../atmosphere-lgetkf/gnssro.yaml.j2 | 10 +- .../satwnd.abi_goes-16.yaml.j2 | 24 +- .../atmosphere-lgetkf/satwnd.ahi_h8.yaml.j2 | 18 +- .../atmosphere-lgetkf/sondes.yaml.j2 | 6 +- .../observations/atmosphere/aircraft.yaml.j2 | 6 +- .../observations/atmosphere/amsua_n19.yaml.j2 | 16 +- .../atmosphere/ascatw.ascat_metop-b.yaml.j2 | 6 +- .../observations/atmosphere/atms_n20.yaml.j2 | 16 +- .../observations/atmosphere/atms_npp.yaml.j2 | 16 +- .../observations/atmosphere/conv_ps.yaml.j2 | 6 +- .../atmosphere/cris-fsr_n20.yaml.j2 | 16 +- .../atmosphere/cris-fsr_npp.yaml.j2 | 16 +- .../observations/atmosphere/gnssro.yaml.j2 | 6 +- .../atmosphere/iasi_metop-a.yaml.j2 | 16 +- .../atmosphere/iasi_metop-b.yaml.j2 | 16 +- .../observations/atmosphere/omi_aura.yaml.j2 | 6 +- .../atmosphere/ompsnp_npp.yaml.j2 | 6 +- .../atmosphere/ompstc_npp.yaml.j2 | 6 +- .../atmosphere/satwnd.abi_goes-16.yaml.j2 | 6 +- .../atmosphere/satwnd.abi_goes-17.yaml.j2 | 6 +- .../atmosphere/satwnd.ahi_h8.yaml.j2 | 6 +- .../atmosphere/satwnd.avhrr_n15.yaml.j2 | 4 +- .../atmosphere/satwnd.avhrr_n18.yaml.j2 | 4 +- .../atmosphere/satwnd.avhrr_n19.yaml.j2 | 4 +- .../atmosphere/satwnd.leogeo_multi.yaml.j2 | 4 +- .../atmosphere/satwnd.modis_aqua.yaml.j2 | 4 +- .../atmosphere/satwnd.modis_terra.yaml.j2 | 4 +- .../atmosphere/satwnd.seviri_m11.yaml.j2 | 6 +- .../atmosphere/satwnd.seviri_m8.yaml.j2 | 6 +- .../atmosphere/satwnd.viirs_n20.yaml.j2 | 4 +- .../atmosphere/satwnd.viirs_npp.yaml.j2 | 4 +- .../atmosphere/scatwind_ascat_metop-a.yaml.j2 | 6 +- .../observations/atmosphere/sfc.yaml.j2 | 6 +- .../observations/atmosphere/sfcship.yaml.j2 | 6 +- .../observations/atmosphere/sondes.yaml.j2 | 6 +- .../observations/atmosphere/ssmis_f17.yaml.j2 | 12 +- .../observations/atmosphere/ssmis_f18.yaml.j2 | 12 +- .../observations/marine/adt_rads_all.yaml.j2 | 52 ++ .../marine/icec_amsr2_north.yaml.j2 | 45 ++ .../marine/icec_amsr2_south.yaml.j2 | 45 ++ .../marine/insitu_profile_argo.yaml.j2 | 603 ++++++++++++++++++ .../marine/insitu_profile_bathy.yaml.j2 | 17 + .../marine/insitu_profile_dbuoy.yaml.j2 | 22 + .../marine/insitu_profile_dbuoyb.yaml.j2 | 22 + .../marine/insitu_profile_glider.yaml.j2 | 22 + .../insitu_profile_marinemammal.yaml.j2 | 22 + .../marine/insitu_profile_mbuoy.yaml.j2 | 22 + .../marine/insitu_profile_mbuoyb.yaml.j2 | 22 + .../marine/insitu_profile_tesac.yaml.j2 | 20 + .../insitu_profile_tesac_salinity.yaml.j2 | 22 + .../marine/insitu_profile_xbtctd.yaml.j2 | 22 + .../marine/insitu_surface_altkob.yaml.j2 | 18 + .../marine/insitu_surface_trkob.yaml.j2 | 18 + .../insitu_surface_trkob_salinity.yaml.j2 | 18 + .../marine/sst_abi_g16_l3c.yaml.j2 | 56 ++ .../marine/sst_abi_g17_l3c.yaml.j2 | 56 ++ .../marine/sst_ahi_h08_l3c.yaml.j2 | 56 ++ .../marine/sst_avhrr_ma_l3u.yaml.j2 | 56 ++ .../marine/sst_avhrr_mb_l3u.yaml.j2 | 56 ++ .../marine/sst_avhrr_mc_l3u.yaml.j2 | 56 ++ .../marine/sst_viirs_n20_l3u.yaml.j2 | 56 ++ .../marine/sst_viirs_npp_l3u.yaml.j2 | 56 ++ 105 files changed, 2071 insertions(+), 402 deletions(-) create mode 100644 parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml rename parm/jcb-gdas/model/aero/{geometry.yaml.j2 => aero_geometry.yaml.j2} (100%) delete mode 100644 parm/jcb-gdas/model/atmosphere/3dvar_outer_loop_1.yaml.j2 delete mode 100644 parm/jcb-gdas/model/atmosphere/3dvar_outer_loop_2.yaml.j2 create mode 100644 parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 create mode 100644 parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 rename parm/jcb-gdas/model/atmosphere/{background.yaml.j2 => atmosphere_background.yaml.j2} (84%) rename parm/jcb-gdas/model/atmosphere/{background_ensemble.yaml.j2 => atmosphere_background_ensemble.yaml.j2} (67%) rename parm/jcb-gdas/model/atmosphere/{background_error_hybrid_bump_bump.yaml.j2 => atmosphere_background_error_hybrid_bump_bump.yaml.j2} (66%) rename parm/jcb-gdas/model/atmosphere/{background_error_hybrid_gsibec_bump.yaml.j2 => atmosphere_background_error_hybrid_gsibec_bump.yaml.j2} (77%) rename parm/jcb-gdas/model/atmosphere/{background_error_static_bump.yaml.j2 => atmosphere_background_error_static_bump.yaml.j2} (68%) create mode 100644 parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_gsibec.yaml.j2 create mode 100644 parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_identity.yaml.j2 create mode 100644 parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_cubed_sphere.yaml.j2 create mode 100644 parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_gaussian.yaml.j2 create mode 100644 parm/jcb-gdas/model/atmosphere/atmosphere_geometry_background.yaml.j2 rename parm/jcb-gdas/model/atmosphere/{model_pseudo.yaml.j2 => atmosphere_model_pseudo.yaml.j2} (86%) rename parm/jcb-gdas/model/atmosphere/{output_ensemble_increments_cubed_sphere.yaml.j2 => atmosphere_output_ensemble_increments_cubed_sphere.yaml.j2} (100%) create mode 100644 parm/jcb-gdas/model/atmosphere/atmosphere_output_ensemble_increments_gaussian.yaml.j2 create mode 100644 parm/jcb-gdas/model/atmosphere/atmosphere_posterior_output_gaussian.yaml.j2 delete mode 100644 parm/jcb-gdas/model/atmosphere/background_error_static_gsibec.yaml.j2 delete mode 100644 parm/jcb-gdas/model/atmosphere/background_error_static_identity.yaml.j2 delete mode 100644 parm/jcb-gdas/model/atmosphere/final_increment_cubed_sphere.yaml.j2 delete mode 100644 parm/jcb-gdas/model/atmosphere/final_increment_gaussian.yaml.j2 delete mode 100644 parm/jcb-gdas/model/atmosphere/geometry_background.yaml.j2 delete mode 100644 parm/jcb-gdas/model/atmosphere/output_ensemble_increments_gaussian.yaml.j2 delete mode 100644 parm/jcb-gdas/model/atmosphere/posterior_output_gaussian.yaml.j2 create mode 100644 parm/jcb-gdas/model/marine/marine_3dfgat_outer_loop_1.yaml.j2 create mode 100644 parm/jcb-gdas/model/marine/marine_3dvar_outer_loop_1.yaml.j2 create mode 100644 parm/jcb-gdas/model/marine/marine_background.yaml.j2 create mode 100644 parm/jcb-gdas/model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 create mode 100644 parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 create mode 100644 parm/jcb-gdas/model/marine/marine_final_increment.yaml.j2 create mode 100644 parm/jcb-gdas/model/marine/marine_geometry_background.yaml.j2 create mode 100644 parm/jcb-gdas/model/marine/marine_model_pseudo.yaml.j2 create mode 100644 parm/jcb-gdas/model/marine/marine_output.yaml.j2 delete mode 100644 parm/jcb-gdas/model/snow/geometry.yaml.j2 rename parm/jcb-gdas/model/{marine/geometry.yaml.j2 => snow/snow_geometry.yaml.j2} (100%) create mode 100644 parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/insitu_profile_argo.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/insitu_profile_dbuoy.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/insitu_profile_dbuoyb.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/insitu_profile_glider.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/insitu_profile_marinemammal.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/insitu_profile_mbuoy.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/insitu_profile_mbuoyb.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/insitu_profile_tesac.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/insitu_profile_tesac_salinity.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/insitu_profile_xbtctd.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/insitu_surface_altkob.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/insitu_surface_trkob.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/insitu_surface_trkob_salinity.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/sst_abi_g16_l3c.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/sst_abi_g17_l3c.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/sst_ahi_h08_l3c.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/sst_avhrr_ma_l3u.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/sst_avhrr_mb_l3u.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/sst_avhrr_mc_l3u.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/sst_viirs_n20_l3u.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/sst_viirs_npp_l3u.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 index eb94c89cb..c7d24cacf 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 @@ -10,11 +10,11 @@ background geometry: field table filename: ./fv3jedi/field_table akbk: ./fv3jedi/akbk.nc4 layout: - - {{ layout_x }} - - {{ layout_y }} - npx: {{ npx_ges }} - npy: {{ npy_ges }} - npz: {{ npz_ges }} + - {{ atmosphere_layout_x }} + - {{ atmosphere_layout_y }} + npx: {{ atmosphere_npx_ges }} + npy: {{ atmosphere_npy_ges }} + npz: {{ atmosphere_npz_ges }} field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_restart.yaml jedi increment geometry: fms initialization: @@ -22,11 +22,11 @@ jedi increment geometry: field table filename: ./fv3jedi/field_table akbk: ./fv3jedi/akbk.nc4 layout: - - {{ layout_x }} - - {{ layout_y }} - npx: {{ npx_ges }} - npy: {{ npy_ges }} - npz: {{ npz_ges }} + - {{ atmosphere_layout_x }} + - {{ atmosphere_layout_y }} + npx: {{ atmosphere_npx_ges }} + npy: {{ atmosphere_npy_ges }} + npz: {{ atmosphere_npz_ges }} field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_restart.yaml fv3 increment geometry: fms initialization: @@ -34,25 +34,25 @@ fv3 increment geometry: field table filename: ./fv3jedi/field_table akbk: ./fv3jedi/akbk.nc4 layout: - - {{ layout_x }} - - {{ layout_y }} - npx: {{ npx_ges }} - npy: {{ npy_ges }} - npz: {{ npz_ges }} + - {{ atmosphere_layout_x }} + - {{ atmosphere_layout_y }} + npx: {{ atmosphere_npx_ges }} + npy: {{ atmosphere_npy_ges }} + npz: {{ atmosphere_npz_ges }} field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_fv3inc.yaml members from template: template: background input: datapath: ./bkg/mem%mem% filetype: fms restart - datetime: '{{ atm_background_time_iso }}' + datetime: '{{ atmosphere_background_time_iso }}' filename is datetime templated: true filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" filename_trcr: "%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc" filename_sfcd: "%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc" filename_sfcw: "%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc" filename_cplr: "%yyyy%mm%dd.%hh%MM%ss.coupler.res" - state variables: *bkgvars + state variables: *bkgvars jedi increment input: filetype: cube sphere history filename: ./anl/mem%mem%/atminc.%yyyy%mm%dd.%hh%MM%ssz.nc4 @@ -62,5 +62,5 @@ members from template: gridtype: gaussian filename: ./anl/mem%mem%/atminc. pattern: '%mem%' - nmembers: {{ number_ensemble_members }} + nmembers: {{ atmosphere_number_ensemble_members }} zero padding: 3 diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 index 5bb86a8ba..63079a2c3 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 @@ -10,11 +10,11 @@ background geometry: field table filename: ./fv3jedi/field_table akbk: ./fv3jedi/akbk.nc4 layout: - - {{ layout_x }} - - {{ layout_y }} - npx: {{ npx_ges }} - npy: {{ npy_ges }} - npz: {{ npz_ges }} + - {{ atmosphere_layout_x }} + - {{ atmosphere_layout_y }} + npx: {{ atmosphere_npx_ges }} + npy: {{ atmosphere_npy_ges }} + npz: {{ atmosphere_npz_ges }} field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_restart.yaml jedi increment geometry: fms initialization: @@ -22,11 +22,11 @@ jedi increment geometry: field table filename: ./fv3jedi/field_table akbk: ./fv3jedi/akbk.nc4 layout: - - {{ layout_x }} - - {{ layout_y }} - npx: {{ npx_anl }} - npy: {{ npy_anl }} - npz: {{ npz_anl }} + - {{ atmosphere_layout_x }} + - {{ atmosphere_layout_y }} + npx: {{ atmosphere_npx_anl }} + npy: {{ atmosphere_npy_anl }} + npz: {{ atmosphere_npz_anl }} field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_history.yaml fv3 increment geometry: fms initialization: @@ -34,17 +34,17 @@ fv3 increment geometry: field table filename: ./fv3jedi/field_table akbk: ./fv3jedi/akbk.nc4 layout: - - {{ layout_x }} - - {{ layout_y }} - npx: {{ npx_anl }} - npy: {{ npy_anl }} - npz: {{ npz_anl }} + - {{ atmosphere_layout_x }} + - {{ atmosphere_layout_y }} + npx: {{ atmosphere_npx_anl }} + npy: {{ atmosphere_npy_anl }} + npz: {{ atmosphere_npz_anl }} field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_fv3inc.yaml members: - background input: datapath: ./bkg filetype: fms restart - datetime: "{{ atm_background_time_iso }}" + datetime: "{{ atmosphere_background_time_iso }}" filename is datetime templated: true filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" filename_trcr: "%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc" diff --git a/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml b/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml new file mode 100644 index 000000000..83ce7048e --- /dev/null +++ b/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml @@ -0,0 +1,28 @@ +geometry: + mom6_input_nml: mom_input.nml + fields metadata: ./fields_metadata.yaml + +date: '{{marine_window_begin}}' + +layers variable: [hocn] + +increment variables: [tocn, socn, uocn, vocn, ssh] + +vertical geometry: + date: '{{marine_window_begin}}' + basename: ./INPUT/ + ocn_filename: MOM.res.nc + read_from_file: 1 + +soca increment: + date: '{{marine_window_begin}}' + basename: ./Data/ + ocn_filename: 'ocn.3dvarfgat_pseudo.incr.{{ATM_WINDOW_MIDDLE}}.nc' + read_from_file: 1 + +output increment: + datadir: ./ + date: '{{marine_window_begin}}' + exp: mom6_iau + type: incr + output file: inc.nc diff --git a/parm/jcb-gdas/model/aero/geometry.yaml.j2 b/parm/jcb-gdas/model/aero/aero_geometry.yaml.j2 similarity index 100% rename from parm/jcb-gdas/model/aero/geometry.yaml.j2 rename to parm/jcb-gdas/model/aero/aero_geometry.yaml.j2 diff --git a/parm/jcb-gdas/model/atmosphere/3dvar_outer_loop_1.yaml.j2 b/parm/jcb-gdas/model/atmosphere/3dvar_outer_loop_1.yaml.j2 deleted file mode 100644 index e50d564c7..000000000 --- a/parm/jcb-gdas/model/atmosphere/3dvar_outer_loop_1.yaml.j2 +++ /dev/null @@ -1,17 +0,0 @@ -- ninner: 2 - gradient norm reduction: 1e-10 - test: on - geometry: - fms initialization: - namelist filename: {{fv3jedi_files_path}}/fmsmpp.nml - field table filename: {{fv3jedi_files_path}}/field_table - akbk: {{fv3jedi_files_path}}/akbk.nc4 - layout: - - {{layout_x}} - - {{layout_y}} - npx: {{npx_anl}} - npy: {{npy_anl}} - npz: {{npz_anl}} - field metadata override: {{fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml - diagnostics: - departures: bkgmob diff --git a/parm/jcb-gdas/model/atmosphere/3dvar_outer_loop_2.yaml.j2 b/parm/jcb-gdas/model/atmosphere/3dvar_outer_loop_2.yaml.j2 deleted file mode 100644 index 1c926f21c..000000000 --- a/parm/jcb-gdas/model/atmosphere/3dvar_outer_loop_2.yaml.j2 +++ /dev/null @@ -1,17 +0,0 @@ -- ninner: 4 - gradient norm reduction: 1e-10 - test: on - geometry: - fms initialization: - namelist filename: {{fv3jedi_files_path}}/fmsmpp.nml - field table filename: {{fv3jedi_files_path}}/field_table - akbk: {{fv3jedi_files_path}}/akbk.nc4 - layout: - - {{layout_x}} - - {{layout_y}} - npx: {{npx_anl}} - npy: {{npy_anl}} - npz: {{npz_anl}} - field metadata override: {{fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml - diagnostics: - departures: bkgmob1 diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 new file mode 100644 index 000000000..9c850f633 --- /dev/null +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 @@ -0,0 +1,17 @@ +- ninner: 2 + gradient norm reduction: 1e-10 + test: on + geometry: + fms initialization: + namelist filename: {{atmosphere_fv3jedi_files_path}}/fmsmpp.nml + field table filename: {{atmosphere_fv3jedi_files_path}}/field_table + akbk: {{atmosphere_fv3jedi_files_path}}/akbk.nc4 + layout: + - {{atmosphere_layout_x}} + - {{atmosphere_layout_y}} + npx: {{atmosphere_npx_anl}} + npy: {{atmosphere_npy_anl}} + npz: {{atmosphere_npz_anl}} + field metadata override: {{atmosphere_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml + diagnostics: + departures: bkgmob diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 new file mode 100644 index 000000000..ec5dc0950 --- /dev/null +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 @@ -0,0 +1,17 @@ +- ninner: 4 + gradient norm reduction: 1e-10 + test: on + geometry: + fms initialization: + namelist filename: {{atmosphere_fv3jedi_files_path}}/fmsmpp.nml + field table filename: {{atmosphere_fv3jedi_files_path}}/field_table + akbk: {{atmosphere_fv3jedi_files_path}}/akbk.nc4 + layout: + - {{atmosphere_layout_x}} + - {{atmosphere_layout_y}} + npx: {{atmosphere_npx_anl}} + npy: {{atmosphere_npy_anl}} + npz: {{atmosphere_npz_anl}} + field metadata override: {{atmosphere_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml + diagnostics: + departures: bkgmob1 diff --git a/parm/jcb-gdas/model/atmosphere/background.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 similarity index 84% rename from parm/jcb-gdas/model/atmosphere/background.yaml.j2 rename to parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 index af8b509c3..e5a882731 100644 --- a/parm/jcb-gdas/model/atmosphere/background.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 @@ -1,6 +1,6 @@ -datapath: {{ background_path}} +datapath: {{ atmosphere_background_path}} filetype: fms restart -datetime: "{{ atm_background_time_iso }}" +datetime: "{{ atmosphere_background_time_iso }}" filename is datetime templated: true filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" filename_trcr: "%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc" diff --git a/parm/jcb-gdas/model/atmosphere/background_ensemble.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background_ensemble.yaml.j2 similarity index 67% rename from parm/jcb-gdas/model/atmosphere/background_ensemble.yaml.j2 rename to parm/jcb-gdas/model/atmosphere/atmosphere_background_ensemble.yaml.j2 index 9b3cf5909..37640ee33 100644 --- a/parm/jcb-gdas/model/atmosphere/background_ensemble.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background_ensemble.yaml.j2 @@ -1,18 +1,18 @@ -datetime: '{{ atm_background_time_iso }}' +datetime: '{{ atmosphere_background_time_iso }}' members from template: template: - datetime: '{{ atm_background_time_iso }}' + datetime: '{{ atmosphere_background_time_iso }}' filetype: fms restart state variables: [ua,va,t,DZ,delp,ps,sphum,ice_wat,liq_wat,o3mr,phis, slmsk,sheleg,tsea,vtype,stype,vfrac,stc,smc,snwdph, u_srf,v_srf,f10m] - datapath: {{ background_ensemble_path}} + datapath: {{ atmosphere_background_ensemble_path}} filename is datetime templated: true filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" filename_trcr: "%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc" filename_sfcd: "%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc" filename_sfcw: "%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc" filename_cplr: "%yyyy%mm%dd.%hh%MM%ss.coupler.res" - pattern: "{{ ensemble_pattern | default("%mem%", true) }}" - nmembers: {{ number_ensemble_members }} + pattern: "{{ atmosphere_ensemble_pattern | default("%mem%", true) }}" + nmembers: {{ atmosphere_number_ensemble_members }} zero padding: 3 diff --git a/parm/jcb-gdas/model/atmosphere/background_error_hybrid_bump_bump.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_bump_bump.yaml.j2 similarity index 66% rename from parm/jcb-gdas/model/atmosphere/background_error_hybrid_bump_bump.yaml.j2 rename to parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_bump_bump.yaml.j2 index 9d3f647df..bf30ace2b 100644 --- a/parm/jcb-gdas/model/atmosphere/background_error_hybrid_bump_bump.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_bump_bump.yaml.j2 @@ -5,12 +5,12 @@ components: saber blocks: - saber block name: BUMP_NICAS saber central block: true - input variables: &control_vars [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] - output variables: *control_vars - active variables: &active_vars [psi,chi,t,ps,rh,liq_wat,o3mr] + input variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] + output variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] + active variables: [psi,chi,t,ps,rh,liq_wat,o3mr] bump: io: - data directory: &staticb_dir {{bump_data_directory}} + data directory: {{atmosphere_bump_data_directory}} drivers: multivariate strategy: specific_univariate read local nicas: true @@ -36,15 +36,15 @@ components: io: files prefix: nicas/nicas_2D_gfs - saber block name: StdDev - input variables: *control_vars - output variables: *control_vars - active variables: *active_vars + input variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] + output variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] + active variables: [psi,chi,t,ps,rh,liq_wat,o3mr] file: filetype: fms restart datetime: 2021-12-22T00:00:00Z set datetime on read: true psinfile: true - datapath: '*staticb_dir' + datapath: {{atmosphere_bump_data_directory}} prefix: stddev/20211222.000000 filename_core: stddev.fv_core.res.nc filename_trcr: stddev.fv_tracer.res.nc @@ -52,14 +52,14 @@ components: filename_sfcw: stddev.fv_srf_wnd.res.nc filename_cplr: stddev.coupler.res - saber block name: BUMP_VerticalBalance - input variables: *control_vars - output variables: *control_vars - active variables: *active_vars + input variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] + output variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] + active variables: [psi,chi,t,ps,rh,liq_wat,o3mr] bump: general: universe length-scale: 2000.0e3 io: - data directory: *staticb_dir + data directory: {{atmosphere_bump_data_directory}} files prefix: vbal/vbal_gfs overriding sampling file: vbal/vbal_2021122200_gfs_sampling drivers: @@ -74,49 +74,49 @@ components: - balanced variable: var4 unbalanced variable: var1 - saber block name: BUMP_PsiChiToUV - input variables: *control_vars - output variables: &3dvars_with_rh [ua,va,t,ps,rh,ice_wat,liq_wat,o3mr] + input variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] + output variables: [ua,va,t,ps,rh,ice_wat,liq_wat,o3mr] active variables: [psi,chi,ua,va] bump: general: universe length-scale: 2000.0e3 io: - data directory: *staticb_dir + data directory: {{atmosphere_bump_data_directory}} files prefix: psichitouv/psichitouv_gfs drivers: read local psichitouv: true linear variable change: linear variable change name: Control2Analysis - input variables: *3dvars_with_rh - output variables: &3dvars [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] + input variables: [ua,va,t,ps,rh,ice_wat,liq_wat,o3mr] + output variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] weight: value: 0.1 - covariance: covariance model: ensemble members from template: template: - datetime: "{{ atm_background_time_iso }}" + datetime: "{{ atmosphere_background_time_iso }}" filetype: fms restart - state variables: *3dvars - datapath: {{ background_ensemble_path}} + state variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] + datapath: {{ atmosphere_background_ensemble_path}} filename is datetime templated: true filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" filename_trcr: "%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc" filename_sfcd: "%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc" filename_sfcw: "%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc" filename_cplr: "%yyyy%mm%dd.%hh%MM%ss.coupler.res" - pattern: "{{ ensemble_pattern | default("%mem%", true) }}" - nmembers: {{number_ensemble_members}} + pattern: "{{ atmosphere_ensemble_pattern | default("%mem%", true) }}" + nmembers: {{atmosphere_number_ensemble_members}} zero padding: 3 # localization: # localization method: SABER # saber block: # - saber block name: BUMP_NICAS -# input variables: *control_vars -# output variables: *control_vars -# active variables: *active_vars +# input variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] +# output variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] +# active variables: [psi,chi,t,ps,rh,liq_wat,o3mr] # bump: -# datadir: *staticb_dir +# datadir: {{atmosphere_bump_data_directory}} # verbosity: main # strategy: specific_univariate # method: loc diff --git a/parm/jcb-gdas/model/atmosphere/background_error_hybrid_gsibec_bump.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 similarity index 77% rename from parm/jcb-gdas/model/atmosphere/background_error_hybrid_gsibec_bump.yaml.j2 rename to parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 index ef8fe88b6..4ba0dbe48 100644 --- a/parm/jcb-gdas/model/atmosphere/background_error_hybrid_gsibec_bump.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 @@ -5,50 +5,50 @@ components: saber central block: saber block name: gsi static covariance read: - gsi akbk: &akbk {{fv3jedi_files_path}}/akbk.nc4 - gsi error covariance file: &gsiberr {{gsibec_path}}/gsi-coeffs-gfs-global.nc4 - gsi berror namelist file: &gsibnml {{gsibec_path}}/gfs_gsi_global.nml - processor layout x direction: &layout_gsib_x {{layout_gsib_x}} - processor layout y direction: &layout_gsib_y {{layout_gsib_y}} + gsi akbk: {{atmosphere_fv3jedi_files_path}}/akbk.nc4 + gsi error covariance file: {{atmosphere_gsibec_path}}/gsi-coeffs-gfs-global.nc4 + gsi berror namelist file: {{atmosphere_gsibec_path}}/gfs_gsi_global.nml + processor layout x direction: {{atmosphere_layout_gsib_x}} + processor layout y direction: {{atmosphere_layout_gsib_y}} debugging mode: false saber outer blocks: - saber block name: gsi interpolation to model grid - gsi akbk: *akbk - gsi error covariance file: *gsiberr - gsi berror namelist file: *gsibnml - processor layout x direction: *layout_gsib_x - processor layout y direction: *layout_gsib_y + gsi akbk: {{atmosphere_fv3jedi_files_path}}/akbk.nc4 + gsi error covariance file: {{atmosphere_gsibec_path}}/gsi-coeffs-gfs-global.nc4 + gsi berror namelist file: {{atmosphere_gsibec_path}}/gfs_gsi_global.nml + processor layout x direction: {{atmosphere_layout_gsib_x}} + processor layout y direction: {{atmosphere_layout_gsib_y}} debugging mode: false linear variable change: linear variable change name: Control2Analysis - input variables: &control_vars [eastward_wind,northward_wind,air_temperature,surface_pressure, + input variables: [eastward_wind,northward_wind,air_temperature,surface_pressure, specific_humidity,cloud_liquid_ice,cloud_liquid_water, mole_fraction_of_ozone_in_air] - output variables: &3dvars_anal [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] + output variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] weight: value: 0.125 - covariance: covariance model: ensemble members from template: template: - datetime: "{{ atm_background_time_iso }}" + datetime: "{{ atmosphere_background_time_iso }}" filetype: fms restart - state variables: *3dvars_anal - datapath: {{ background_ensemble_path}} + state variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] + datapath: {{ atmosphere_background_ensemble_path}} filename is datetime templated: true filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" filename_trcr: "%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc" filename_sfcd: "%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc" filename_sfcw: "%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc" filename_cplr: "%yyyy%mm%dd.%hh%MM%ss.coupler.res" - pattern: "{{ ensemble_pattern | default("%mem%", true) }}" - nmembers: {{number_ensemble_members}} + pattern: "{{ atmosphere_ensemble_pattern | default("%mem%", true) }}" + nmembers: {{atmosphere_number_ensemble_members}} zero padding: 3 localization: localization method: SABER saber central block: saber block name: BUMP_NICAS - active variables: &3dvars_anal_long [eastward_wind,northward_wind,air_temperature,surface_pressure, + active variables: [eastward_wind,northward_wind,air_temperature,surface_pressure, specific_humidity,cloud_liquid_ice,cloud_liquid_water, ozone_mass_mixing_ratio ] read: @@ -201,7 +201,9 @@ components: value: 0.3 linear variable change: linear variable change name: Control2Analysis - input variables: *3dvars_anal_long - output variables: *3dvars_anal + input variables: [eastward_wind,northward_wind,air_temperature,surface_pressure, + specific_humidity,cloud_liquid_ice,cloud_liquid_water, + ozone_mass_mixing_ratio ] + output variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] weight: value: 0.875 diff --git a/parm/jcb-gdas/model/atmosphere/background_error_static_bump.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_bump.yaml.j2 similarity index 68% rename from parm/jcb-gdas/model/atmosphere/background_error_static_bump.yaml.j2 rename to parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_bump.yaml.j2 index 65be183d6..68601e969 100644 --- a/parm/jcb-gdas/model/atmosphere/background_error_static_bump.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_bump.yaml.j2 @@ -3,12 +3,12 @@ full inverse: true saber blocks: - saber block name: BUMP_NICAS saber central block: true - input variables: &control_vars [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] - output variables: *control_vars - active variables: &active_vars [psi,chi,t,ps,rh,liq_wat,o3mr] + input variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] + output variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] + active variables: [psi,chi,t,ps,rh,liq_wat,o3mr] bump: io: - data directory: &staticb_dir {{bump_data_directory}} + data directory: {{atmosphere_bump_data_directory}} drivers: multivariate strategy: specific_univariate read local nicas: true @@ -34,15 +34,15 @@ saber blocks: io: files prefix: nicas/nicas_2D_gfs - saber block name: StdDev - input variables: *control_vars - output variables: *control_vars - active variables: *active_vars + input variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] + output variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] + active variables: [psi,chi,t,ps,rh,liq_wat,o3mr] file: filetype: fms restart datetime: 2021-12-22T00:00:00Z set datetime on read: true psinfile: true - datapath: *staticb_dir + datapath: {{atmosphere_bump_data_directory}} prefix: stddev/20211222.000000 filename_core: stddev.fv_core.res.nc filename_trcr: stddev.fv_tracer.res.nc @@ -50,14 +50,14 @@ saber blocks: filename_sfcw: stddev.fv_srf_wnd.res.nc filename_cplr: stddev.coupler.res - saber block name: BUMP_VerticalBalance - input variables: *control_vars - output variables: *control_vars - active variables: *active_vars + input variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] + output variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] + active variables: [psi,chi,t,ps,rh,liq_wat,o3mr] bump: general: universe length-scale: 2000.0e3 io: - data directory: *staticb_dir + data directory: {{atmosphere_bump_data_directory}} files prefix: vbal/vbal_gfs overriding sampling file: vbal/vbal_2021122200_gfs_sampling drivers: @@ -72,18 +72,18 @@ saber blocks: - balanced variable: var4 unbalanced variable: var1 - saber block name: BUMP_PsiChiToUV - input variables: *control_vars - output variables: &3dvars_with_rh [ua,va,t,ps,rh,ice_wat,liq_wat,o3mr] + input variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] + output variables: [ua,va,t,ps,rh,ice_wat,liq_wat,o3mr] active variables: [psi,chi,ua,va] bump: general: universe length-scale: 2000.0e3 io: - data directory: '*staticb_dir' + data directory: {{atmosphere_bump_data_directory}} files prefix: psichitouv/psichitouv_gfs drivers: read local psichitouv: true linear variable change: linear variable change name: Control2Analysis - input variables: *3dvars_with_rh + input variables: [ua,va,t,ps,rh,ice_wat,liq_wat,o3mr] output variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_gsibec.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_gsibec.yaml.j2 new file mode 100644 index 000000000..242ce4f2e --- /dev/null +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_gsibec.yaml.j2 @@ -0,0 +1,25 @@ +covariance model: SABER +full inverse: true +saber central block: + saber block name: gsi static covariance + read: + gsi akbk: {{atmosphere_fv3jedi_files_path}}/akbk.nc4 + gsi error covariance file: {{atmosphere_gsibec_path}}/gsi-coeffs-gfs-global.nc4 + gsi berror namelist file: {{atmosphere_gsibec_path}}/gfs_gsi_global.nml + processor layout x direction: {{atmosphere_layout_gsib_x}} + processor layout y direction: {{atmosphere_layout_gsib_y}} + debugging mode: false +saber outer blocks: +- saber block name: gsi interpolation to model grid + gsi akbk: {{atmosphere_fv3jedi_files_path}}/akbk.nc4 + gsi error covariance file: {{atmosphere_gsibec_path}}/gsi-coeffs-gfs-global.nc4 + gsi berror namelist file: {{atmosphere_gsibec_path}}/gfs_gsi_global.nml + processor layout x direction: {{atmosphere_layout_gsib_x}} + processor layout y direction: {{atmosphere_layout_gsib_y}} + debugging mode: false +linear variable change: + linear variable change name: Control2Analysis + input variables: [eastward_wind,northward_wind,air_temperature,surface_pressure, + specific_humidity,cloud_liquid_ice,cloud_liquid_water, + mole_fraction_of_ozone_in_air] + output variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_identity.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_identity.yaml.j2 new file mode 100644 index 000000000..a2533ec23 --- /dev/null +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_identity.yaml.j2 @@ -0,0 +1,2 @@ +covariance model: FV3JEDI-ID +date: "{{ atmosphere_background_time_iso }}" diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_cubed_sphere.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_cubed_sphere.yaml.j2 new file mode 100644 index 000000000..7d9f7e7e0 --- /dev/null +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_cubed_sphere.yaml.j2 @@ -0,0 +1,18 @@ +output: + state component: + filetype: cube sphere history + filename: ./anl/atminc.%yyyy%mm%dd.%hh%MM%ssz.nc4 + provider: ufs + fields to write: [ugrd,vgrd,tmp,pressfc,spfh,icmr,clwmr,o3mr] +geometry: + fms initialization: + namelist filename: "{{atmosphere_fv3jedi_files_path}}/fmsmpp.nml" + field table filename: "{{atmosphere_fv3jedi_files_path}}/field_table" + akbk: "{{atmosphere_fv3jedi_files_path}}/akbk.nc4" + layout: + - {{atmosphere_layout_x}} + - {{atmosphere_layout_y}} + npx: {{atmosphere_npx_anl}} + npy: {{atmosphere_npy_anl}} + npz: {{atmosphere_npz_anl}} + field metadata override: "{{atmosphere_fv3jedi_files_path}}/fv3jedi_fieldmetadata_history.yaml" diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_gaussian.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_gaussian.yaml.j2 new file mode 100644 index 000000000..1d3bc4c53 --- /dev/null +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_gaussian.yaml.j2 @@ -0,0 +1,17 @@ +output: + state component: + filetype: auxgrid + gridtype: gaussian + filename: {{atmosphere_final_increment_prefix}} +geometry: + fms initialization: + namelist filename: "{{atmosphere_fv3jedi_files_path}}/fmsmpp.nml" + field table filename: "{{atmosphere_fv3jedi_files_path}}/field_table" + akbk: "{{atmosphere_fv3jedi_files_path}}/akbk.nc4" + layout: + - {{atmosphere_layout_x}} + - {{atmosphere_layout_y}} + npx: {{atmosphere_npx_anl}} + npy: {{atmosphere_npy_anl}} + npz: {{atmosphere_npz_anl}} + field metadata override: "{{atmosphere_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_geometry_background.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_geometry_background.yaml.j2 new file mode 100644 index 000000000..65ec71bbf --- /dev/null +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_geometry_background.yaml.j2 @@ -0,0 +1,11 @@ +fms initialization: + namelist filename: "{{atmosphere_fv3jedi_files_path}}/fmsmpp.nml" + field table filename: "{{atmosphere_fv3jedi_files_path}}/field_table" +akbk: "{{atmosphere_fv3jedi_files_path}}/akbk.nc4" +layout: +- {{atmosphere_layout_x}} +- {{atmosphere_layout_y}} +npx: {{atmosphere_npx_ges}} +npy: {{atmosphere_npy_ges}} +npz: {{atmosphere_npz_ges}} +field metadata override: "{{atmosphere_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" diff --git a/parm/jcb-gdas/model/atmosphere/model_pseudo.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_model_pseudo.yaml.j2 similarity index 86% rename from parm/jcb-gdas/model/atmosphere/model_pseudo.yaml.j2 rename to parm/jcb-gdas/model/atmosphere/atmosphere_model_pseudo.yaml.j2 index 05eefc26c..cf054e369 100644 --- a/parm/jcb-gdas/model/atmosphere/model_pseudo.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_model_pseudo.yaml.j2 @@ -1,5 +1,5 @@ name: PSEUDO -datapath: {{ background_path}} +datapath: {{ atmosphere_background_path}} filetype: fms restart filename is datetime templated: true filename_core: '%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc' @@ -7,7 +7,7 @@ filename_trcr: '%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc' filename_sfcd: '%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc' filename_sfcw: '%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc' filename_cplr: '%yyyy%mm%dd.%hh%MM%ss.coupler.res' -tstep: {{forecast_timestep}} +tstep: {{atmosphere_forecast_timestep}} model variables: [ua,va,t,delp,sphum,ice_wat,liq_wat,o3mr,phis, slmsk,sheleg,tsea,vtype,stype,vfrac,stc,smc,snwdph, u_srf,v_srf,f10m] diff --git a/parm/jcb-gdas/model/atmosphere/output_ensemble_increments_cubed_sphere.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_output_ensemble_increments_cubed_sphere.yaml.j2 similarity index 100% rename from parm/jcb-gdas/model/atmosphere/output_ensemble_increments_cubed_sphere.yaml.j2 rename to parm/jcb-gdas/model/atmosphere/atmosphere_output_ensemble_increments_cubed_sphere.yaml.j2 diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_output_ensemble_increments_gaussian.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_output_ensemble_increments_gaussian.yaml.j2 new file mode 100644 index 000000000..fc6674f55 --- /dev/null +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_output_ensemble_increments_gaussian.yaml.j2 @@ -0,0 +1,3 @@ +filetype: auxgrid +gridtype: gaussian +filename: {{atmosphere_ensemble_increment_prefix}} diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_posterior_output_gaussian.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_posterior_output_gaussian.yaml.j2 new file mode 100644 index 000000000..512d2350f --- /dev/null +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_posterior_output_gaussian.yaml.j2 @@ -0,0 +1,3 @@ +filetype: auxgrid +gridtype: gaussian +filename: {{atmosphere_posterior_output_gaussian}} diff --git a/parm/jcb-gdas/model/atmosphere/background_error_static_gsibec.yaml.j2 b/parm/jcb-gdas/model/atmosphere/background_error_static_gsibec.yaml.j2 deleted file mode 100644 index e53b8fd9d..000000000 --- a/parm/jcb-gdas/model/atmosphere/background_error_static_gsibec.yaml.j2 +++ /dev/null @@ -1,25 +0,0 @@ -covariance model: SABER -full inverse: true -saber central block: - saber block name: gsi static covariance - read: - gsi akbk: &akbk {{fv3jedi_files_path}}/akbk.nc4 - gsi error covariance file: &gsiberr {{gsibec_path}}/gsi-coeffs-gfs-global.nc4 - gsi berror namelist file: &gsibnml {{gsibec_path}}/gfs_gsi_global.nml - processor layout x direction: &layout_gsib_x {{layout_gsib_x}} - processor layout y direction: &layout_gsib_y {{layout_gsib_y}} - debugging mode: false -saber outer blocks: -- saber block name: gsi interpolation to model grid - gsi akbk: *akbk - gsi error covariance file: *gsiberr - gsi berror namelist file: *gsibnml - processor layout x direction: *layout_gsib_x - processor layout y direction: *layout_gsib_y - debugging mode: false -linear variable change: - linear variable change name: Control2Analysis - input variables: &control_vars [eastward_wind,northward_wind,air_temperature,surface_pressure, - specific_humidity,cloud_liquid_ice,cloud_liquid_water, - mole_fraction_of_ozone_in_air] - output variables: &3dvars_anal [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] diff --git a/parm/jcb-gdas/model/atmosphere/background_error_static_identity.yaml.j2 b/parm/jcb-gdas/model/atmosphere/background_error_static_identity.yaml.j2 deleted file mode 100644 index c5e6c99d8..000000000 --- a/parm/jcb-gdas/model/atmosphere/background_error_static_identity.yaml.j2 +++ /dev/null @@ -1,2 +0,0 @@ -covariance model: FV3JEDI-ID -date: "{{ atm_background_time_iso }}" diff --git a/parm/jcb-gdas/model/atmosphere/final_increment_cubed_sphere.yaml.j2 b/parm/jcb-gdas/model/atmosphere/final_increment_cubed_sphere.yaml.j2 deleted file mode 100644 index 247810768..000000000 --- a/parm/jcb-gdas/model/atmosphere/final_increment_cubed_sphere.yaml.j2 +++ /dev/null @@ -1,18 +0,0 @@ -output: - state component: - filetype: cube sphere history - filename: ./anl/atminc.%yyyy%mm%dd.%hh%MM%ssz.nc4 - provider: ufs - fields to write: [ugrd,vgrd,tmp,pressfc,spfh,icmr,clwmr,o3mr] -geometry: - fms initialization: - namelist filename: "{{fv3jedi_files_path}}/fmsmpp.nml" - field table filename: "{{fv3jedi_files_path}}/field_table" - akbk: "{{fv3jedi_files_path}}/akbk.nc4" - layout: - - {{layout_x}} - - {{layout_y}} - npx: {{npx_anl}} - npy: {{npy_anl}} - npz: {{npz_anl}} - field metadata override: "{{fv3jedi_files_path}}/fv3jedi_fieldmetadata_history.yaml" diff --git a/parm/jcb-gdas/model/atmosphere/final_increment_gaussian.yaml.j2 b/parm/jcb-gdas/model/atmosphere/final_increment_gaussian.yaml.j2 deleted file mode 100644 index a57243361..000000000 --- a/parm/jcb-gdas/model/atmosphere/final_increment_gaussian.yaml.j2 +++ /dev/null @@ -1,17 +0,0 @@ -output: - state component: - filetype: auxgrid - gridtype: gaussian - filename: {{final_increment_prefix}} -geometry: - fms initialization: - namelist filename: "{{fv3jedi_files_path}}/fmsmpp.nml" - field table filename: "{{fv3jedi_files_path}}/field_table" - akbk: "{{fv3jedi_files_path}}/akbk.nc4" - layout: - - {{layout_x}} - - {{layout_y}} - npx: {{npx_anl}} - npy: {{npy_anl}} - npz: {{npz_anl}} - field metadata override: "{{fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" diff --git a/parm/jcb-gdas/model/atmosphere/geometry_background.yaml.j2 b/parm/jcb-gdas/model/atmosphere/geometry_background.yaml.j2 deleted file mode 100644 index fefe71c1b..000000000 --- a/parm/jcb-gdas/model/atmosphere/geometry_background.yaml.j2 +++ /dev/null @@ -1,11 +0,0 @@ -fms initialization: - namelist filename: "{{fv3jedi_files_path}}/fmsmpp.nml" - field table filename: "{{fv3jedi_files_path}}/field_table" -akbk: "{{fv3jedi_files_path}}/akbk.nc4" -layout: -- {{layout_x}} -- {{layout_y}} -npx: {{npx_ges}} -npy: {{npy_ges}} -npz: {{npz_ges}} -field metadata override: "{{fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" diff --git a/parm/jcb-gdas/model/atmosphere/output_ensemble_increments_gaussian.yaml.j2 b/parm/jcb-gdas/model/atmosphere/output_ensemble_increments_gaussian.yaml.j2 deleted file mode 100644 index d11ca4db2..000000000 --- a/parm/jcb-gdas/model/atmosphere/output_ensemble_increments_gaussian.yaml.j2 +++ /dev/null @@ -1,3 +0,0 @@ -filetype: auxgrid -gridtype: gaussian -filename: {{ensemble_increment_prefix}} diff --git a/parm/jcb-gdas/model/atmosphere/posterior_output_gaussian.yaml.j2 b/parm/jcb-gdas/model/atmosphere/posterior_output_gaussian.yaml.j2 deleted file mode 100644 index 471137f4a..000000000 --- a/parm/jcb-gdas/model/atmosphere/posterior_output_gaussian.yaml.j2 +++ /dev/null @@ -1,3 +0,0 @@ -filetype: auxgrid -gridtype: gaussian -filename: {{posterior_output_gaussian}} diff --git a/parm/jcb-gdas/model/marine/marine_3dfgat_outer_loop_1.yaml.j2 b/parm/jcb-gdas/model/marine/marine_3dfgat_outer_loop_1.yaml.j2 new file mode 100644 index 000000000..2931f62a1 --- /dev/null +++ b/parm/jcb-gdas/model/marine/marine_3dfgat_outer_loop_1.yaml.j2 @@ -0,0 +1,7 @@ +- ninner: {{marine_ninner_1}} + gradient norm reduction: {{marine_grad_red_1}} + geometry: + mom6_input_nml: mom_input.nml + fields metadata: {{marine_soca_files_path}}/fields_metadata.yaml + diagnostics: + departures: ombg diff --git a/parm/jcb-gdas/model/marine/marine_3dvar_outer_loop_1.yaml.j2 b/parm/jcb-gdas/model/marine/marine_3dvar_outer_loop_1.yaml.j2 new file mode 100644 index 000000000..2931f62a1 --- /dev/null +++ b/parm/jcb-gdas/model/marine/marine_3dvar_outer_loop_1.yaml.j2 @@ -0,0 +1,7 @@ +- ninner: {{marine_ninner_1}} + gradient norm reduction: {{marine_grad_red_1}} + geometry: + mom6_input_nml: mom_input.nml + fields metadata: {{marine_soca_files_path}}/fields_metadata.yaml + diagnostics: + departures: ombg diff --git a/parm/jcb-gdas/model/marine/marine_background.yaml.j2 b/parm/jcb-gdas/model/marine/marine_background.yaml.j2 new file mode 100644 index 000000000..d0047e85e --- /dev/null +++ b/parm/jcb-gdas/model/marine/marine_background.yaml.j2 @@ -0,0 +1,6 @@ +read_from_file: 1 +basename: {{ marine_background_path}} +ocn_filename: MOM.res.nc +ice_filename: cice.res.nc +date: "{{ marine_background_time }}" +state variables: [cicen, hicen, hsnon, socn, tocn, uocn, vocn, ssh, hocn, mld, layer_depth] diff --git a/parm/jcb-gdas/model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 b/parm/jcb-gdas/model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 new file mode 100644 index 000000000..43feedb7b --- /dev/null +++ b/parm/jcb-gdas/model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 @@ -0,0 +1,92 @@ +covariance model: hybrid +components: +- covariance: + covariance model: SABER + saber central block: + saber block name: EXPLICIT_DIFFUSION + active variables: [tocn, socn, ssh, cicen] + geometry: + mom6_input_nml: mom_input.nml + fields metadata: ./fields_metadata.yaml + group mapping: + - name: ocean + variables: + - tocn + - socn + - ssh + - name: ice + variables: + - cicen + read: + groups: + - name: ocean + horizontal: + filename: hz_ocean.nc + vertical: + filename: vt_ocean.nc + - name: ice + horizontal: + filename: hz_ice.nc + + saber outer blocks: + - saber block name: StdDev + read: + model file: + date: '{{marine_window_middle}}' + basename: ./ + ocn_filename: 'ocn.bkgerr_stddev.incr.{{marine_window_middle}}.nc' + ice_filename: 'ice.bkgerr_stddev.incr.{{marine_window_middle}}.nc' + read_from_file: 3 + + linear variable change: + input variables: [cicen, hicen, hsnon, socn, tocn, uocn, vocn, ssh] + output variables: [cicen, hicen, hsnon, socn, tocn, uocn, vocn, ssh] + linear variable changes: + - linear variable change name: BalanceSOCA + + weight: + value: 1.00 + +- covariance: + covariance model: ensemble + members from template: + template: + read_from_file: 1 + date: '{{marine_window_middle}}' + basename: ./static_ens/ + ocn_filename: 'ocn.pert.steric.%mem%.nc' + ice_filename: 'ice.%mem%.nc' + state variables: [tocn, socn, ssh, uocn, vocn, cicen, hicen, hsnon] + pattern: '%mem%' + nmembers: '{{marine_number_ensemble_members}}' + localization: + localization method: SABER + saber central block: + saber block name: EXPLICIT_DIFFUSION + active variables: [tocn, socn, ssh] + geometry: + mom6_input_nml: mom_input.nml + fields metadata: ./fields_metadata.yaml + group mapping: + - name: ocean + variables: [tocn, socn, ssh, uocn, vocn] + - name: ice + variables: [cicen, hicen, hsnon] + read: + groups: + - name: ocean + multivariate strategy: duplicated + horizontal: + filename: hz_ocean.nc + vertical: + strategy: duplicated + - name: ice + horizontal: + filename: hz_ice.nc + + weight: + read_from_file: 3 + basename: ./ + ocn_filename: 'ocn.ens_weights.incr.{{marine_window_middle}}.nc' + ice_filename: 'ice.ens_weights.incr.{{marine_window_middle}}.nc' + date: '{{marine_window_middle}}' diff --git a/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 b/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 new file mode 100644 index 000000000..a2d7e0547 --- /dev/null +++ b/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 @@ -0,0 +1,42 @@ +covariance model: SABER +saber central block: + saber block name: EXPLICIT_DIFFUSION + active variables: [tocn, socn, ssh, cicen] + geometry: + mom6_input_nml: mom_input.nml + fields metadata: ./fields_metadata.yaml + group mapping: + - name: ocean + variables: + - tocn + - socn + - ssh + - name: ice + variables: + - cicen + read: + groups: + - name: ocean + horizontal: + filename: hz_ocean.nc + vertical: + filename: vt_ocean.nc + - name: ice + horizontal: + filename: hz_ice.nc + +saber outer blocks: +- saber block name: StdDev + read: + model file: + date: '{{marine_stddev_time}}' + basename: ./ + ocn_filename: 'ocn.bkgerr_stddev.incr.{{marine_stddev_time}}.nc' + ice_filename: 'ice.bkgerr_stddev.incr.{{marine_stddev_time}}.nc' + read_from_file: 3 + +linear variable change: + input variables: [cicen, hicen, hsnon, socn, tocn, uocn, vocn, ssh] + output variables: [cicen, hicen, hsnon, socn, tocn, uocn, vocn, ssh] + linear variable changes: + - linear variable change name: BalanceSOCA diff --git a/parm/jcb-gdas/model/marine/marine_final_increment.yaml.j2 b/parm/jcb-gdas/model/marine/marine_final_increment.yaml.j2 new file mode 100644 index 000000000..ead7a3ed1 --- /dev/null +++ b/parm/jcb-gdas/model/marine/marine_final_increment.yaml.j2 @@ -0,0 +1,9 @@ +output: + state component: + datadir: {{marine_final_increment_dir}} + date: '{{marine_window_begin}}' + exp: {{marine_final_increment_exp}} + type: incr +geometry: + mom6_input_nml: mom_input.nml + fields metadata: {{marine_soca_files_path}}/fields_metadata.yaml diff --git a/parm/jcb-gdas/model/marine/marine_geometry_background.yaml.j2 b/parm/jcb-gdas/model/marine/marine_geometry_background.yaml.j2 new file mode 100644 index 000000000..2ab3b6789 --- /dev/null +++ b/parm/jcb-gdas/model/marine/marine_geometry_background.yaml.j2 @@ -0,0 +1,2 @@ +mom6_input_nml: mom_input.nml +fields metadata: {{marine_soca_files_path}}/fields_metadata.yaml diff --git a/parm/jcb-gdas/model/marine/marine_model_pseudo.yaml.j2 b/parm/jcb-gdas/model/marine/marine_model_pseudo.yaml.j2 new file mode 100644 index 000000000..9188340bb --- /dev/null +++ b/parm/jcb-gdas/model/marine/marine_model_pseudo.yaml.j2 @@ -0,0 +1,10 @@ +name: PseudoModel +tstep: {{marine_forecast_timestep}} +states: +{% for marine_pseudo_model_state in marine_pseudo_model_states %} +- date: '{{ marine_pseudo_model_state.date }}' + basename: '{{ marine_pseudo_model_state.basename }}' + ocn_filename: '{{ marine_pseudo_model_state.ocn_filename }}' + ice_filename: '{{ marine_pseudo_model_state.ice_filename }}' + read_from_file: {{ marine_pseudo_model_state.read_from_file }} +{% endfor %} diff --git a/parm/jcb-gdas/model/marine/marine_output.yaml.j2 b/parm/jcb-gdas/model/marine/marine_output.yaml.j2 new file mode 100644 index 000000000..2c8e2adb9 --- /dev/null +++ b/parm/jcb-gdas/model/marine/marine_output.yaml.j2 @@ -0,0 +1,4 @@ +datadir: {{marine_output_dir}} +exp: {{marine_output_exp}} +type: an +frequency: {{marine_output_freq}} diff --git a/parm/jcb-gdas/model/snow/geometry.yaml.j2 b/parm/jcb-gdas/model/snow/geometry.yaml.j2 deleted file mode 100644 index e69de29bb..000000000 diff --git a/parm/jcb-gdas/model/marine/geometry.yaml.j2 b/parm/jcb-gdas/model/snow/snow_geometry.yaml.j2 similarity index 100% rename from parm/jcb-gdas/model/marine/geometry.yaml.j2 rename to parm/jcb-gdas/model/snow/snow_geometry.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 index eff404a12..15f4846da 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 @@ -10,11 +10,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [brightnessTemperature] @@ -40,14 +40,14 @@ # Observation Bias Correction (VarBC) # ----------------------------------- obs bias: - input file: "{{atm_obsbiasin_path}}/{{atm_obsbiasin_prefix}}{{observation_from_jcb}}{{atm_obsbiasin_suffix}}" - output file: "{{atm_obsbiasout_path}}/{{atm_obsbiasout_prefix}}{{observation_from_jcb}}{{atm_obsbiasout_suffix}}" + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" variational bc: predictors: - name: constant - name: lapseRate order: 2 - tlapse: &{{observation_from_jcb}}_tlapse "{{atm_obsbiasin_path}}/{{atm_obstlapsein_prefix}}{{observation_from_jcb}}{{atm_obstlapsein_suffix}}" + tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" - name: lapseRate tlapse: *{{observation_from_jcb}}_tlapse - name: emissivityJacobian @@ -64,11 +64,11 @@ step size: 1.0e-4 largest analysis variance: 10000.0 prior: - input file: "{{atm_obsbiasin_path}}/{{atm_obsbiascovin_prefix}}{{observation_from_jcb}}{{atm_obsbiascovin_suffix}}" + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiascovin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovin_suffix}}" inflation: ratio: 1.1 ratio for small dataset: 2.0 - output file: "{{atm_obsbiasout_path}}/{{atm_obsbiascovout_prefix}}{{observation_from_jcb}}{{atm_obsbiascovout_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" # Observation Filters (QC) # ------------------------ @@ -101,7 +101,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/ascatw.ascat_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/ascatw.ascat_metop-b.yaml.j2 index a35d6dce7..788ead9ed 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/ascatw.ascat_metop-b.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/ascatw.ascat_metop-b.yaml.j2 @@ -10,11 +10,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [windEastward, windNorthward] @@ -276,7 +276,7 @@ # this inflation to take place both nvqc must be .true. AND ibeta must be >0, see: # https://github.com/NOAA-EMC/GSI/blob/14ae595af1b03471287d322596d35c0665336e95/src/gsi/setupw.f90#L1229 # GSI settings must have ibeta>0 for satwinds, but not for scatwinds. - # + # # If the ibeta settings for scatwinds were to change while nvqc remained .true., we would extend YAML to # an additional filter that inflates final ob-errors across-the-board by 1/0.8 = 1.25. NOTE: the nvqc setting # is defaulted to .false. in GSI code, but is overridden in global operational configuration. See: @@ -316,7 +316,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/conv_ps.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/conv_ps.yaml.j2 index 014fa5269..0ca33b1f2 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/conv_ps.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/conv_ps.yaml.j2 @@ -10,11 +10,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [stationPressure] @@ -40,7 +40,7 @@ # Observation Prior Filters (QC) # ------------------------------ obs prior filters: - # Initial Error Assignments for SFC Observations + # Initial Error Assignments for SFC Observations - filter: Perform Action filter variables: - name: stationPressure @@ -73,7 +73,7 @@ name: ObsValue/stationPressure xvals: [85000, 80000, 75000, 70000, 65000, 60000, 55000] errors: [120, 140, 140, 140, 140, 140, 1.0e+11] - # Initial Error Assignments for SFCSHIP Observations + # Initial Error Assignments for SFCSHIP Observations - filter: Perform Action filter variables: - name: stationPressure @@ -124,7 +124,7 @@ type: float source variable: ObsErrorData/stationPressure - # Set observation quality-realted variables + # Set observation quality-realted variables # Create PreQC group variable (pqm in GSI read_prepbufr) - filter: Variable Assignment assignments: @@ -203,7 +203,7 @@ # - variable: # name: ObsType/stationPressure # is_in: 183 - ## Reject surface pressure below 500 hPa + ## Reject surface pressure below 500 hPa #- filter: Bounds Check # filter variables: # - name: stationPressure @@ -241,7 +241,7 @@ variable: stationPressure # Reduce effective observation error based on obs type and subtype - # In this case: reduce effective obs error for buoy + # In this case: reduce effective obs error for buoy - filter: Perform Action filter variables: - name: stationPressure @@ -256,7 +256,7 @@ name: inflate error inflation factor: 0.7 - # Calculate obs error inflation factors for large discrepancies between model and observations + # Calculate obs error inflation factors for large discrepancies between model and observations - filter: Variable Assignment assignments: - name: ObsErrorFactorSfcPressure/stationPressure @@ -344,7 +344,7 @@ inflation variable: name: ObsErrorFactorDuplicateCheck/stationPressure - # Reject data based on PreUseFlag (usage in GSI) + # Reject data based on PreUseFlag (usage in GSI) - filter: Perform Action filter variables: - name: stationPressure @@ -353,7 +353,7 @@ is_not_in: 0, 1 action: name: reject - # End of Filters + # End of Filters # Observation Localizations (LocalEnsembleDA) # ------------------------------------------- @@ -366,7 +366,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/gnssro.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/gnssro.yaml.j2 index 5075a201a..b26d992a5 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/gnssro.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/gnssro.yaml.j2 @@ -10,7 +10,7 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" # obsgrouping: # group variables: [ "sequenceNumber" ] # sort variable: "impactHeightRO" @@ -18,7 +18,7 @@ obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" simulated variables: [bendingAngle] # Observation Operator @@ -35,7 +35,7 @@ # ------------------------ obs filters: # Apply gross check using pccf - # Step 0-A: Create Diagnostic Flags + # Step 0-A: Create Diagnostic Flags # Diagnostic flag for pccf - filter: Create Diagnostic Flags filter variables: @@ -126,7 +126,7 @@ minvalue: 8000.1 action: name: reject - #4. assign obs error + #4. assign obs error - filter: ROobserror filter variables: - name: bendingAngle @@ -160,7 +160,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-16.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-16.yaml.j2 index 17f601c3c..1b727ad1c 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-16.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-16.yaml.j2 @@ -10,11 +10,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [windEastward, windNorthward] @@ -40,7 +40,7 @@ Transform: SurfaceWindScalingPressure SkipWhenNoObs: false - # Calculate error inflation factor for duplicate observations + # Calculate error inflation factor for duplicate observations #- filter: Variable Assignment # assignments: # - name: ObsErrorFactorDuplicateCheck/windEastward @@ -191,7 +191,7 @@ name: reject # GSI read routine QC (part-1) - # Exclude Type 251 (VIS) --- obs tossed without passing to setup routine + # Exclude Type 251 (VIS) --- obs tossed without passing to setup routine - filter: Perform Action filter variables: - name: windEastward @@ -227,7 +227,7 @@ action: name: reject - # Exclude data over non-water surface type where latitude > 20N for Type 240 (IRSW) and Type 245 (IRLW) --- obs tossed and not passed to setup routine + # Exclude data over non-water surface type where latitude > 20N for Type 240 (IRSW) and Type 245 (IRLW) --- obs tossed and not passed to setup routine # Notes: This check was missing, so added (eliu) # Replace land_type_index_NPOSS with water_area_fraction (eliu) - filter: Bounds Check @@ -374,7 +374,7 @@ # GSI setupw routine QC # Reject any ob Type [240–260] when pressure greater than 950 mb. - # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa - filter: Bounds Check filter variables: - name: windEastward @@ -420,8 +420,8 @@ # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged - # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation - # Turn this check off for now. + # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation + # Turn this check off for now. # Need to check if troposphere pressure was implemented correctly in fv3-jed - filter: Difference Check filter variables: @@ -433,7 +433,7 @@ action: name: reject - # GOES (247) reject any observation with a /=0 surface type (non-water + # GOES (247) reject any observation with a /=0 surface type (non-water # surface) within 110 hPa of the surface pressure (as part of the LNVD # check). # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested # Notes (eliu): Replace land_type_index_NPOESS with land_area_fraction. @@ -570,7 +570,7 @@ # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. - # + # # If this nvqc functionality were to be switched off (i.e. if variational qc were to be turned off), # you would want to remove this last inflation filter. #- filter: Perform Action @@ -591,7 +591,7 @@ # action: # name: inflate error # inflation factor: 1.25 - # End of Filters + # End of Filters # Observation Localizations (LocalEnsembleDA) # ------------------------------------------- @@ -604,7 +604,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.ahi_h8.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.ahi_h8.yaml.j2 index 7664fcc57..a745ee1e3 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.ahi_h8.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.ahi_h8.yaml.j2 @@ -10,11 +10,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [windEastward, windNorthward] @@ -53,7 +53,7 @@ Transform: SurfaceWindScalingPressure SkipWhenNoObs: false - # Calculate error inflation factor for duplicate observations + # Calculate error inflation factor for duplicate observations #- filter: Variable Assignment # assignments: # - name: ObsErrorFactorDuplicateCheck/windEastward @@ -227,7 +227,7 @@ # GSI setupw routine QC # Reject any ob Type [240–260] when pressure greater than 950 mb. - # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa - filter: Bounds Check filter variables: - name: windEastward @@ -287,8 +287,8 @@ # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged - # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation - # Turn this check off for now. + # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation + # Turn this check off for now. # Need to check if troposphere pressure was implemented correctly in fv3-jed - filter: Difference Check filter variables: @@ -401,7 +401,7 @@ # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. - # + # # If this nvqc functionality were to be switched off (i.e. if variational qc were to be turned off), # you would want to remove this last inflation filter. #- filter: Perform Action @@ -422,7 +422,7 @@ # action: # name: inflate error # inflation factor: 1.25 - # End of Filters + # End of Filters # Observation Localizations (LocalEnsembleDA) # ------------------------------------------- @@ -435,7 +435,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/sondes.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/sondes.yaml.j2 index 650be3296..0bc719339 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/sondes.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/sondes.yaml.j2 @@ -10,11 +10,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [windEastward, windNorthward, airTemperature] @@ -52,7 +52,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/aircraft.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/aircraft.yaml.j2 index 22b65c043..f5e081719 100644 --- a/parm/jcb-gdas/observations/atmosphere/aircraft.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/aircraft.yaml.j2 @@ -7,7 +7,7 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsgrouping: group variables: ["stationIdentification"] sort variable: "pressure" @@ -15,7 +15,7 @@ obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [windEastward, windNorthward, airTemperature, specificHumidity] @@ -349,7 +349,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 index d8dd23f28..dba4162db 100644 --- a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 @@ -7,11 +7,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [brightnessTemperature] @@ -32,14 +32,14 @@ # Observation Bias Correction (VarBC) # ----------------------------------- obs bias: - input file: "{{atm_obsbiasin_path}}/{{atm_obsbiasin_prefix}}{{observation_from_jcb}}{{atm_obsbiasin_suffix}}" - output file: "{{atm_obsbiasout_path}}/{{atm_obsbiasout_prefix}}{{observation_from_jcb}}{{atm_obsbiasout_suffix}}" + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" variational bc: predictors: - name: constant - name: lapseRate order: 2 - tlapse: &{{observation_from_jcb}}_tlapse "{{atm_obsbiasin_path}}/{{atm_obstlapsein_prefix}}{{observation_from_jcb}}{{atm_obstlapsein_suffix}}" + tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" - name: lapseRate tlapse: *{{observation_from_jcb}}_tlapse - name: emissivityJacobian @@ -56,11 +56,11 @@ step size: 1.0e-4 largest analysis variance: 10000.0 prior: - input file: "{{atm_obsbiasin_path}}/{{atm_obsbiascovin_prefix}}{{observation_from_jcb}}{{atm_obsbiascovin_suffix}}" + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiascovin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovin_suffix}}" inflation: ratio: 1.1 ratio for small dataset: 2.0 - output file: "{{atm_obsbiasout_path}}/{{atm_obsbiascovout_prefix}}{{observation_from_jcb}}{{atm_obsbiascovout_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" # Observation Filters (QC) # ------------------------ @@ -339,7 +339,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/ascatw.ascat_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ascatw.ascat_metop-b.yaml.j2 index dfcfc6067..bb3d5de74 100644 --- a/parm/jcb-gdas/observations/atmosphere/ascatw.ascat_metop-b.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ascatw.ascat_metop-b.yaml.j2 @@ -7,11 +7,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [windEastward, windNorthward] @@ -306,7 +306,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 index fe492cf20..839821963 100644 --- a/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 @@ -7,11 +7,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [brightnessTemperature] @@ -35,14 +35,14 @@ # Observation Bias Correction (VarBC) # ----------------------------------- obs bias: - input file: "{{atm_obsbiasin_path}}/{{atm_obsbiasin_prefix}}{{observation_from_jcb}}{{atm_obsbiasin_suffix}}" - output file: "{{atm_obsbiasout_path}}/{{atm_obsbiasout_prefix}}{{observation_from_jcb}}{{atm_obsbiasout_suffix}}" + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" variational bc: predictors: - name: constant - name: lapse_rate order: 2 - tlapse: &{{observation_from_jcb}}_tlapse "{{atm_obsbiasin_path}}/{{atm_obstlapsein_prefix}}{{observation_from_jcb}}{{atm_obstlapsein_suffix}}" + tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" - name: lapse_rate tlapse: *{{observation_from_jcb}}_tlapse - name: emissivity @@ -59,11 +59,11 @@ step size: 1.0e-4 largest analysis variance: 10000.0 prior: - input file: "{{atm_obsbiasin_path}}/{{atm_obsbiascovin_prefix}}{{observation_from_jcb}}{{atm_obsbiascovin_suffix}}" + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiascovin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovin_suffix}}" inflation: ratio: 1.1 ratio for small dataset: 2.0 - output file: "{{atm_obsbiasout_path}}/{{atm_obsbiascovout_prefix}}{{observation_from_jcb}}{{atm_obsbiascovout_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" # Observation Pre Filters (QC) # ---------------------------- @@ -530,7 +530,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 index 196bbf471..7c68b3e89 100644 --- a/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 @@ -7,11 +7,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [brightnessTemperature] @@ -35,14 +35,14 @@ # Observation Bias Correction (VarBC) # ----------------------------------- obs bias: - input file: "{{atm_obsbiasin_path}}/{{atm_obsbiasin_prefix}}{{observation_from_jcb}}{{atm_obsbiasin_suffix}}" - output file: "{{atm_obsbiasout_path}}/{{atm_obsbiasout_prefix}}{{observation_from_jcb}}{{atm_obsbiasout_suffix}}" + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" variational bc: predictors: - name: constant - name: lapse_rate order: 2 - tlapse: &{{observation_from_jcb}}_tlapse "{{atm_obsbiasin_path}}/{{atm_obstlapsein_prefix}}{{observation_from_jcb}}{{atm_obstlapsein_suffix}}" + tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" - name: lapse_rate tlapse: *{{observation_from_jcb}}_tlapse - name: emissivity @@ -59,11 +59,11 @@ step size: 1.0e-4 largest analysis variance: 10000.0 prior: - input file: "{{atm_obsbiasin_path}}/{{atm_obsbiascovin_prefix}}{{observation_from_jcb}}{{atm_obsbiascovin_suffix}}" + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiascovin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovin_suffix}}" inflation: ratio: 1.1 ratio for small dataset: 2.0 - output file: "{{atm_obsbiasout_path}}/{{atm_obsbiascovout_prefix}}{{observation_from_jcb}}{{atm_obsbiascovout_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" # Observation Pre Filters (QC) # ---------------------------- @@ -548,7 +548,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/conv_ps.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/conv_ps.yaml.j2 index ea52885bf..d8d90f9c8 100644 --- a/parm/jcb-gdas/observations/atmosphere/conv_ps.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/conv_ps.yaml.j2 @@ -7,11 +7,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [stationPressure] @@ -356,7 +356,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 index 547e820d0..9e1fb87cf 100644 --- a/parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 @@ -7,11 +7,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [brightness_temperature] @@ -32,14 +32,14 @@ # Observation Bias Correction (VarBC) # ----------------------------------- obs bias: - input file: "{{atm_obsbiasin_path}}/{{atm_obsbiasin_prefix}}{{observation_from_jcb}}{{atm_obsbiasin_suffix}}" - output file: "{{atm_obsbiasout_path}}/{{atm_obsbiasout_prefix}}{{observation_from_jcb}}{{atm_obsbiasout_suffix}}" + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" variational bc: predictors: - name: constant - name: lapse_rate order: 2 - tlapse: &{{observation_from_jcb}}_tlapse "{{atm_obsbiasin_path}}/{{atm_obstlapsein_prefix}}{{observation_from_jcb}}{{atm_obstlapsein_suffix}}" + tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" - name: lapse_rate tlapse: *{{observation_from_jcb}}_tlapse - name: emissivity @@ -56,11 +56,11 @@ step size: 1.0e-4 largest analysis variance: 10000.0 prior: - input file: "{{atm_obsbiasin_path}}/{{atm_obsbiascovin_prefix}}{{observation_from_jcb}}{{atm_obsbiascovin_suffix}}" + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiascovin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovin_suffix}}" inflation: ratio: 1.1 ratio for small dataset: 2.0 - output file: "{{atm_obsbiasout_path}}/{{atm_obsbiascovout_prefix}}{{observation_from_jcb}}{{atm_obsbiascovout_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" # #obs filters: ## Wavenumber Check @@ -439,7 +439,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/cris-fsr_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/cris-fsr_npp.yaml.j2 index 90194caf8..be21f7863 100644 --- a/parm/jcb-gdas/observations/atmosphere/cris-fsr_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/cris-fsr_npp.yaml.j2 @@ -7,11 +7,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [brightness_temperature] @@ -32,14 +32,14 @@ # Observation Bias Correction (VarBC) # ----------------------------------- obs bias: - input file: "{{atm_obsbiasin_path}}/{{atm_obsbiasin_prefix}}{{observation_from_jcb}}{{atm_obsbiasin_suffix}}" - output file: "{{atm_obsbiasout_path}}/{{atm_obsbiasout_prefix}}{{observation_from_jcb}}{{atm_obsbiasout_suffix}}" + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" variational bc: predictors: - name: constant - name: lapse_rate order: 2 - tlapse: &{{observation_from_jcb}}_tlapse "{{atm_obsbiasin_path}}/{{atm_obstlapsein_prefix}}{{observation_from_jcb}}{{atm_obstlapsein_suffix}}" + tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" - name: lapse_rate tlapse: *{{observation_from_jcb}}_tlapse - name: emissivity @@ -56,11 +56,11 @@ step size: 1.0e-4 largest analysis variance: 10000.0 prior: - input file: "{{atm_obsbiasin_path}}/{{atm_obsbiascovin_prefix}}{{observation_from_jcb}}{{atm_obsbiascovin_suffix}}" + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiascovin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovin_suffix}}" inflation: ratio: 1.1 ratio for small dataset: 2.0 - output file: "{{atm_obsbiasout_path}}/{{atm_obsbiascovout_prefix}}{{observation_from_jcb}}{{atm_obsbiascovout_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" #obs filters: ## Wavenumber Check #- filter: BlackList @@ -438,7 +438,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro.yaml.j2 index 3a984c0c2..6aab75efa 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro.yaml.j2 @@ -7,7 +7,7 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" # obsgrouping: # group variables: [ "sequenceNumber" ] # sort variable: "impactHeightRO" @@ -15,7 +15,7 @@ obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" simulated variables: [bendingAngle] # Observation Operator @@ -150,7 +150,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 index b546ac5b3..141ddfc0a 100644 --- a/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 @@ -7,11 +7,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 observed variables: [radiance] @@ -34,14 +34,14 @@ # Observation Bias Correction (VarBC) # ----------------------------------- obs bias: - input file: "{{atm_obsbiasin_path}}/{{atm_obsbiasin_prefix}}{{observation_from_jcb}}{{atm_obsbiasin_suffix}}" - output file: "{{atm_obsbiasout_path}}/{{atm_obsbiasout_prefix}}{{observation_from_jcb}}{{atm_obsbiasout_suffix}}" + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" variational bc: predictors: - name: constant - name: lapse_rate order: 2 - tlapse: &{{observation_from_jcb}}_tlapse "{{atm_obsbiasin_path}}/{{atm_obstlapsein_prefix}}{{observation_from_jcb}}{{atm_obstlapsein_suffix}}" + tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" - name: lapse_rate tlapse: *{{observation_from_jcb}}_tlapse - name: emissivity @@ -58,11 +58,11 @@ step size: 1.0e-4 largest analysis variance: 10000.0 prior: - input file: "{{atm_obsbiasin_path}}/{{atm_obsbiascovin_prefix}}{{observation_from_jcb}}{{atm_obsbiascovin_suffix}}" + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiascovin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovin_suffix}}" inflation: ratio: 1.1 ratio for small dataset: 2.0 - output file: "{{atm_obsbiasout_path}}/{{atm_obsbiascovout_prefix}}{{observation_from_jcb}}{{atm_obsbiascovout_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" # Observation Pre Filters (QC) # ---------------------------- @@ -572,7 +572,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 index 51696bd96..051d62109 100644 --- a/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 @@ -7,11 +7,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 observed variables: [radiance] @@ -34,14 +34,14 @@ # Observation Bias Correction (VarBC) # ----------------------------------- obs bias: - input file: "{{atm_obsbiasin_path}}/{{atm_obsbiasin_prefix}}{{observation_from_jcb}}{{atm_obsbiasin_suffix}}" - output file: "{{atm_obsbiasout_path}}/{{atm_obsbiasout_prefix}}{{observation_from_jcb}}{{atm_obsbiasout_suffix}}" + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" variational bc: predictors: - name: constant - name: lapse_rate order: 2 - tlapse: &{{observation_from_jcb}}_tlapse "{{atm_obsbiasin_path}}/{{atm_obstlapsein_prefix}}{{observation_from_jcb}}{{atm_obstlapsein_suffix}}" + tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" - name: lapse_rate tlapse: *{{observation_from_jcb}}_tlapse - name: emissivity @@ -58,11 +58,11 @@ step size: 1.0e-4 largest analysis variance: 10000.0 prior: - input file: "{{atm_obsbiasin_path}}/{{atm_obsbiascovin_prefix}}{{observation_from_jcb}}{{atm_obsbiascovin_suffix}}" + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiascovin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovin_suffix}}" inflation: ratio: 1.1 ratio for small dataset: 2.0 - output file: "{{atm_obsbiasout_path}}/{{atm_obsbiascovout_prefix}}{{observation_from_jcb}}{{atm_obsbiascovout_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" # Observation Pre Filters (QC) # ---------------------------- @@ -572,7 +572,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/omi_aura.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/omi_aura.yaml.j2 index 3b5cd8dd4..d8b6d1be0 100644 --- a/parm/jcb-gdas/observations/atmosphere/omi_aura.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/omi_aura.yaml.j2 @@ -7,11 +7,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [ozoneTotal] @@ -168,7 +168,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/ompsnp_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ompsnp_npp.yaml.j2 index 069771ba6..807eb9478 100644 --- a/parm/jcb-gdas/observations/atmosphere/ompsnp_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ompsnp_npp.yaml.j2 @@ -7,7 +7,7 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsgrouping: group variables: ["latitude"] sort variable: "pressure" @@ -15,7 +15,7 @@ obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [ozoneLayer] @@ -323,7 +323,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/ompstc_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ompstc_npp.yaml.j2 index 81c64c8d7..c6e388471 100644 --- a/parm/jcb-gdas/observations/atmosphere/ompstc_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ompstc_npp.yaml.j2 @@ -7,11 +7,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [ozoneTotal] @@ -128,7 +128,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 index 21c0f49f5..e01f17ce4 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 @@ -7,11 +7,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [windEastward, windNorthward] @@ -594,7 +594,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-17.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-17.yaml.j2 index bc4b072e4..b0a41796f 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-17.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-17.yaml.j2 @@ -7,11 +7,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [windEastward, windNorthward] @@ -594,7 +594,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h8.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h8.yaml.j2 index f7569d38f..0aac154e6 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h8.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h8.yaml.j2 @@ -7,11 +7,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [windEastward, windNorthward] @@ -425,7 +425,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n15.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n15.yaml.j2 index 7fe70cdd3..bbe81a8c7 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n15.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n15.yaml.j2 @@ -3,11 +3,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [windEastward, windNorthward] diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n18.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n18.yaml.j2 index fee0d851f..fcced81ff 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n18.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n18.yaml.j2 @@ -3,11 +3,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [windEastward, windNorthward] diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n19.yaml.j2 index 6af10f99e..efb84712f 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n19.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n19.yaml.j2 @@ -3,11 +3,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [windEastward, windNorthward] diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.leogeo_multi.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.leogeo_multi.yaml.j2 index b267c33a4..11ca76e10 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.leogeo_multi.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.leogeo_multi.yaml.j2 @@ -3,11 +3,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [windEastward, windNorthward] diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.modis_aqua.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.modis_aqua.yaml.j2 index 4ab84f56b..d0af7524d 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.modis_aqua.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.modis_aqua.yaml.j2 @@ -3,11 +3,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [windEastward, windNorthward] diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.modis_terra.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.modis_terra.yaml.j2 index 811da8211..0f6d3f773 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.modis_terra.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.modis_terra.yaml.j2 @@ -3,11 +3,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [windEastward, windNorthward] diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m11.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m11.yaml.j2 index 58e85d6a1..cb6a10dc0 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m11.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m11.yaml.j2 @@ -7,11 +7,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [windEastward, windNorthward] @@ -432,7 +432,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m8.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m8.yaml.j2 index aac8a659e..e85bb815a 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m8.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m8.yaml.j2 @@ -7,11 +7,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [windEastward, windNorthward] @@ -432,7 +432,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_n20.yaml.j2 index 033d1e5b3..67949e6ab 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_n20.yaml.j2 @@ -3,11 +3,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [windEastward, windNorthward] diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_npp.yaml.j2 index c8148fbbb..6528a59e5 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_npp.yaml.j2 @@ -3,11 +3,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [windEastward, windNorthward] diff --git a/parm/jcb-gdas/observations/atmosphere/scatwind_ascat_metop-a.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/scatwind_ascat_metop-a.yaml.j2 index 10cca6889..0c1a74514 100644 --- a/parm/jcb-gdas/observations/atmosphere/scatwind_ascat_metop-a.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/scatwind_ascat_metop-a.yaml.j2 @@ -7,11 +7,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [windEastward, windNorthward] @@ -306,7 +306,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/sfc.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/sfc.yaml.j2 index 3192c2acd..b470a0eac 100644 --- a/parm/jcb-gdas/observations/atmosphere/sfc.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/sfc.yaml.j2 @@ -7,11 +7,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [stationPressure] @@ -33,7 +33,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/sfcship.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/sfcship.yaml.j2 index 2c5e25e12..5e1a227d4 100644 --- a/parm/jcb-gdas/observations/atmosphere/sfcship.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/sfcship.yaml.j2 @@ -7,12 +7,12 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File overwrite: true - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [stationPressure, airTemperature, specificHumidity] @@ -93,7 +93,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 index f56414440..aff438b96 100644 --- a/parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 @@ -7,7 +7,7 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsgrouping: group variables: ["stationIdentification"] sort variable: "pressure" @@ -15,7 +15,7 @@ obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 simulated variables: [stationPressure, airTemperature, windEastward, windNorthward, @@ -121,7 +121,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 index c89cb2767..763f27467 100644 --- a/parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 @@ -7,11 +7,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" simulated variables: [brightness_temperature] channels: {{ get_satellite_variable(observation_from_jcb, "simulated") }} @@ -28,8 +28,8 @@ # Observation Bias Correction (VarBC) # ----------------------------------- obs bias: - input file: "{{atm_obsbiasin_path}}/{{atm_obsbiasin_prefix}}{{observation_from_jcb}}{{atm_obsbiasin_suffix}}" - output file: "{{atm_obsbiasout_path}}/{{atm_obsbiasout_prefix}}{{observation_from_jcb}}{{atm_obsbiasout_suffix}}" + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" variational bc: predictors: - name: constant @@ -46,7 +46,7 @@ - name: sine_of_latitude - name: lapse_rate order: 2 - tlapse: &{{observation_from_jcb}}_tlapse "{{atm_obsbiasin_path}}/{{atm_obstlapsein_prefix}}{{observation_from_jcb}}{{atm_obstlapsein_suffix}}" + tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" - name: lapse_rate tlapse: *{{observation_from_jcb}}_tlapse - name: emissivity @@ -170,7 +170,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 index 5cd86df70..4f55848a3 100644 --- a/parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 @@ -7,11 +7,11 @@ obsdatain: engine: type: H5File - obsfile: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}{{atm_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" obsdataout: engine: type: H5File - obsfile: "{{atm_obsdataout_path}}/{{atm_obsdataout_prefix}}{{observation_from_jcb}}{{atm_obsdataout_suffix}}" + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" simulated variables: [brightness_temperature] channels: {{ get_satellite_variable(observation_from_jcb, "simulated") }} @@ -28,8 +28,8 @@ # Observation Bias Correction (VarBC) # ----------------------------------- obs bias: - input file: "{{atm_obsbiasin_path}}/{{atm_obsbiasin_prefix}}{{observation_from_jcb}}{{atm_obsbiasin_suffix}}" - output file: "{{atm_obsbiasout_path}}/{{atm_obsbiasout_prefix}}{{observation_from_jcb}}{{atm_obsbiasout_suffix}}" + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" variational bc: predictors: - name: constant @@ -46,7 +46,7 @@ - name: sine_of_latitude - name: lapse_rate order: 2 - tlapse: &{{observation_from_jcb}}_tlapse "{{atm_obsbiasin_path}}/{{atm_obstlapsein_prefix}}{{observation_from_jcb}}{{atm_obstlapsein_suffix}}" + tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" - name: lapse_rate tlapse: *{{observation_from_jcb}}_tlapse - name: emissivity @@ -170,7 +170,7 @@ # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: - filename: "{{atm_obsdatain_path}}/{{atm_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atm_obsdatain_suffix}}" + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" # Passed benchmark for UFO testing # -------------------------------- diff --git a/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 b/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 new file mode 100644 index 000000000..da532ecaf --- /dev/null +++ b/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 @@ -0,0 +1,52 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [absoluteDynamicTopography] + io pool: + max pool size: 1 + get values: + time interpolation: linear + obs operator: + name: ADT + obs error: + covariance model: diagonal + obs filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + minvalue: 0.9 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_surface_temperature} + minvalue: 5.0 + - filter: Background Check + absolute threshold: 1.0 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_floor_depth_below_sea_surface} + minvalue: 500 + - filter: Perform Action + action: + name: assign error + error function: + name: ObsFunction/LinearCombination + options: + variables: [GeoVaLs/mesoscale_representation_error, + ObsError/absoluteDynamicTopography] + coefs: [0.1, + 1.0] + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_ice_area_fraction} + maxvalue: 0.00001 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/distance_from_coast} + minvalue: 100e3 diff --git a/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 new file mode 100644 index 000000000..c63389837 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 @@ -0,0 +1,45 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaIceFraction] + io pool: + max pool size: 1 + get values: + time interpolation: linear + obs operator: + name: Identity + observation alias file: obsop_name_map.yaml + obs error: + covariance model: diagonal + obs filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + minvalue: 0.9 + - filter: Bounds Check + minvalue: 0.0 + maxvalue: 1.0 + #- filter: Background Check + # threshold: 5.0 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_surface_temperature} + maxvalue: 2.0 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_surface_temperature} + maxvalue: 0.0 + action: + name: inflate error + inflation factor: 2.0 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/distance_from_coast} + minvalue: 100e3 diff --git a/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 new file mode 100644 index 000000000..c63389837 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 @@ -0,0 +1,45 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaIceFraction] + io pool: + max pool size: 1 + get values: + time interpolation: linear + obs operator: + name: Identity + observation alias file: obsop_name_map.yaml + obs error: + covariance model: diagonal + obs filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + minvalue: 0.9 + - filter: Bounds Check + minvalue: 0.0 + maxvalue: 1.0 + #- filter: Background Check + # threshold: 5.0 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_surface_temperature} + maxvalue: 2.0 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_surface_temperature} + maxvalue: 0.0 + action: + name: inflate error + inflation factor: 2.0 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/distance_from_coast} + minvalue: 100e3 diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_argo.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_argo.yaml.j2 new file mode 100644 index 000000000..4a1dbd7bc --- /dev/null +++ b/parm/jcb-gdas/observations/marine/insitu_profile_argo.yaml.j2 @@ -0,0 +1,603 @@ +- obs space: + name: insitu_profile_argo + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsgrouping: + group variables: [latitude, longitude, dateTime] + sort variable: depth + sort group: MetaData + sort order: ascending + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [waterTemperature, salinity] + observed variables: [waterTemperature, salinity] + derived variables: [waterPressure] + io pool: + max pool size: 1 + obs operator: + name: Composite + components: + - name: InsituTemperature + variables: + - name: waterTemperature + - name: VertInterp + # note: alias file is needed for now because of a conflict with "waterTemperature". + # The built-in alias maps to "ocean_temperature", but soca uses "sea_water_temperature" + observation alias file: obsop_name_map.yaml + variables: + - name: salinity + vertical coordinate: sea_water_depth + observation vertical coordinate: depth + interpolation method: linear + obs error: + covariance model: diagonal + + #------------------------------------------------------------------------------- + # START OF OBS FILTERS (work done by Kriti Bhargava) + # The QC filters used here are based on the document by IODE that can be found at + # https://cdn.ioos.noaa.gov/media/2017/12/recommendations_in_situ_data_real_time_qc.pdf + #------------------------------------------------------------------------------- + obs filters: + + # land check + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + minvalue: 0.5 + + #------------------------------------------------------------------------------- + ## Filters for T: + #------------------------------------------------------------------------------- + #------------------------------------------------------------------------------- + ### Global range test + #----------------------------------------------------------------------------- + - filter: Bounds Check + filter variables: [{name: waterTemperature}] + minvalue: -2.5 + maxvalue: 40.0 + + #----------------------------------------------------------------------------- + ### Regional range tests + #----------------------------------------------------------------------------- + + #### Red Sea + #----------------------------------------------------------------------------- + #### + #### the document linked here describes Red sea as the are between 10N, 40E; + #### 20N, 50E; 30N, 30E; 10N, 40E. But that would also include Gulf of Aden. + #### A more reasonable choice seemed to be a box that extends from 10 N to + #### 30 N and 30 E to 45 East . + + - filter: Bounds Check + filter variables: [{name: waterTemperature}] + minvalue: 21.7 + maxvalue: 40.0 + where: + - variable: + name: MetaData/latitude + minvalue: 10 + maxvalue: 30 + - variable: + name: MetaData/longitude + minvalue: 30 + maxvalue: 45 + + #### Mediterranean Sea + #----------------------------------------------------------------------------- + ##### Area 1/3 for Mediterranean Sea + - filter: Bounds Check + filter variables: [{name: waterTemperature}] + minvalue: 10.0 + maxvalue: 40.0 + where: + - variable: + name: MetaData/latitude + minvalue: 30 + maxvalue: 40 + - variable: + name: MetaData/longitude + minvalue: -6 + maxvalue: 40 + ##### Area 2/3 for Mediterranean Sea + - filter: Bounds Check + filter variables: [{name: waterTemperature}] + minvalue: 10.0 + maxvalue: 40.0 + where: + - variable: + name: MetaData/latitude + minvalue: 40 + maxvalue: 41.5 + - variable: + name: MetaData/longitude + minvalue: 20 + maxvalue: 30 + ##### Area 3/3 for Mediterranean Sea + - filter: Bounds Check + filter variables: [{name: waterTemperature}] + minvalue: 10.0 + maxvalue: 40.0 + where: + - variable: + name: MetaData/latitude + minvalue: 40 + maxvalue: 46 + - variable: + name: MetaData/longitude + minvalue: 0 + maxvalue: 20 + + #### Northwestern shelves + #----------------------------------------------------------------------------- + - filter: Bounds Check + filter variables: [{name: waterTemperature}] + minvalue: -2.0 + maxvalue: 24.0 + where: + - variable: + name: MetaData/latitude + minvalue: 50 + maxvalue: 60 + - variable: + name: MetaData/longitude + minvalue: -20 + maxvalue: 10 + #### Southwestern shelves + #----------------------------------------------------------------------------- + - filter: Bounds Check + filter variables: [{name: waterTemperature}] + minvalue: -2.0 + maxvalue: 30 + where: + - variable: + name: MetaData/latitude + minvalue: 25 + maxvalue: 50 + - variable: + name: MetaData/longitude + minvalue: -30 + maxvalue: 0 + + #### Arctic Sea + #----------------------------------------------------------------------------- + - filter: Bounds Check + filter variables: [{name: waterTemperature}] + minvalue: -1.92 + maxvalue: 25.0 + where: + - variable: + name: MetaData/latitude + minvalue: 60 + + - filter: Background Check + filter variables: [{name: waterTemperature}] + threshold: 5.0 + absolute threshold: 5.0 + + #------------------------------------------------------------------------------- + ### Spike test + #----------------------------------------------------------------------------- + - filter: Create Diagnostic Flags + filter variables: + - name: waterTemperature + - name: salinity + flags: + - name: spike + initial value: false + - name: step + initial value: false + + - filter: Spike and Step Check + filter variables: + - name: ObsValue/waterTemperature + dependent: ObsValue/waterTemperature # dy/ + independent: MetaData/depth # dx + count spikes: true + count steps: true + tolerance: + nominal value: 10 # K nominal, in the case of temperature (not really) + gradient: 0.1 # K/m - if dy/dx greater, could be a spike + gradient x resolution: 10 # m - can't know dx to better precision + factors: [1,1,0.5] # multiply tolerance, for ranges bounded by... + x boundaries: [0,500,500] # ...these values of x (depth in m) + boundary layer: + x range: [0.0,300.0] # when bounded by these x values (depth in m)... + step tolerance range: [0.0,-2.0] # ...relax tolerance for steps in boundary layer... + maximum x interval: [50.0,100.0] # ...and ignore level if dx greater than this + action: + name: reject + + #### Count spikes + #----------------------------------------------------------------------------- + - filter: Variable Assignment # create derived obs value containing only T spikes + assignments: + - name: DerivedMetaData/waterTemperature_spikes + type: int + function: + name: IntObsFunction/ProfileLevelCount + options: + where: + - variable: + name: DiagnosticFlags/spike/waterTemperature + value: is_true + + #### Count steps + #----------------------------------------------------------------------------- + - filter: Variable Assignment # create derived obs value containing only T steps + assignments: + - name: DerivedMetaData/waterTemperature_steps + type: int + function: + name: IntObsFunction/ProfileLevelCount + options: + where: + - variable: + name: DiagnosticFlags/step/waterTemperature + value: is_true + #### Count total rejections + #----------------------------------------------------------------------------- + - filter: Variable Assignment # compute sum 2*spikes+steps + assignments: + - name: DerivedMetaData/waterTemperature_rejections + type: int + function: + name: IntObsFunction/LinearCombination + options: + variables: [DerivedMetaData/waterTemperature_spikes, DerivedMetaData/waterTemperature_steps] + coefs: [2,1] + #### Reject entire profile if total rejctions > threshold + #----------------------------------------------------------------------------- + - filter: Perform Action # reject whole profile if 2*spikes+steps>=rejection threshold + where: + - variable: + name: DerivedMetaData/waterTemperature_rejections + minvalue: 3 + action: + name: reject + + #------------------------------------------------------------------------------- + ## Filters for S: + #------------------------------------------------------------------------------- + #----------------------------------------------------------------------------- + ### Global range test + #----------------------------------------------------------------------------- + - filter: Bounds Check + filter variables: [{name: salinity}] + minvalue: 2.0 + maxvalue: 41.0 + + #----------------------------------------------------------------------------- + ### Regional range test + #----------------------------------------------------------------------------- + #### Red Sea + #----------------------------------------------------------------------------- + #### + #### the document linked here describes Red sea as the are between 10N, 40E; + #### 20N, 50E; 30N, 30E; 10N, 40E. But that would also include Gulf of Aden. + #### A more reasonable choice seemed to be a box that extends from 10 N to + #### 30 N and 30 E to 45 East . + + - filter: Bounds Check + filter variables: [{name: salinity}] + minvalue: 2.0 + maxvalue: 41.0 + where: + - variable: + name: MetaData/latitude + minvalue: 10 + maxvalue: 30 + - variable: + name: MetaData/longitude + minvalue: 30 + maxvalue: 45 + + #### Mediterranean Sea + #----------------------------------------------------------------------------- + ##### Area 1/3 for Mediterranean Sea + - filter: Bounds Check + filter variables: [{name: salinity}] + minvalue: 2.0 + maxvalue: 40.0 + where: + - variable: + name: MetaData/latitude + minvalue: 30 + maxvalue: 40 + - variable: + name: MetaData/longitude + minvalue: -6 + maxvalue: 40 + ##### Area 2/3 for Mediterranean Sea + - filter: Bounds Check + filter variables: [{name: salinity}] + minvalue: 2.0 + maxvalue: 40.0 + where: + - variable: + name: MetaData/latitude + minvalue: 40 + maxvalue: 41.5 + - variable: + name: MetaData/longitude + minvalue: 20 + maxvalue: 30 + ##### Area 3/3 for Mediterranean Sea + - filter: Bounds Check + filter variables: [{name: salinity}] + minvalue: 2.0 + maxvalue: 40.0 + where: + - variable: + name: MetaData/latitude + minvalue: 40 + maxvalue: 46 + - variable: + name: MetaData/longitude + minvalue: 0 + maxvalue: 20 + + + #### Northwestern shelves + #----------------------------------------------------------------------------- + - filter: Bounds Check + filter variables: [{name: salinity}] + minvalue: 0.0 + maxvalue: 37.0 + where: + - variable: + name: MetaData/latitude + minvalue: 50 + maxvalue: 60 + - variable: + name: MetaData/longitude + minvalue: -20 + maxvalue: 10 + + #### Southwestern shelves + #----------------------------------------------------------------------------- + - filter: Bounds Check + filter variables: [{name: salinity}] + minvalue: 0.0 + maxvalue: 38 + where: + - variable: + name: MetaData/latitude + minvalue: 25 + maxvalue: 50 + - variable: + name: MetaData/longitude + minvalue: -30 + maxvalue: 0 + + #### Arctic Sea + #----------------------------------------------------------------------------- + - filter: Bounds Check + filter variables: [{name: salinity}] + minvalue: 2.0 + maxvalue: 40.0 + where: + - variable: + name: MetaData/latitude + minvalue: 60 + + - filter: Background Check + filter variables: [{name: salinity}] + threshold: 5.0 + absolute threshold: 5.0 + + #------------------------------------------------------------------------------- + ### Spike test + #----------------------------------------------------------------------------- + - filter: Spike and Step Check + filter variables: + - name: ObsValue/salinity + dependent: ObsValue/salinity # dy/ + independent: MetaData/depth # dx + count spikes: true + count steps: true + tolerance: + nominal value: 1.0 # PSU nominal, in the case of salinity (not really) + threshold: 0.6 # weird salinity thing + factors: [1,1,0.2] # multiply tolerance, for ranges bounded by... + x boundaries: [0,200,600] # ...these values of x (depth in m) + boundary layer: + x range: [0.0,300.0] # when bounded by these x values (depth in m)... + maximum x interval: [50.0,100.0] # ...and ignore level if dx greater than this + action: + name: reject + + #### Count spikes + #----------------------------------------------------------------------------- + - filter: Variable Assignment # create derived obs value containing only S spikes + assignments: + - name: DerivedMetaData/salinity_spikes + type: int + function: + name: IntObsFunction/ProfileLevelCount + options: + where: + - variable: + name: DiagnosticFlags/spike/salinity + value: is_true + #### Count steps + #----------------------------------------------------------------------------- + - filter: Variable Assignment # create derived obs value containing only S steps + assignments: + - name: DerivedMetaData/salinity_steps + type: int + function: + name: IntObsFunction/ProfileLevelCount + options: + where: + - variable: + name: DiagnosticFlags/step/salinity + value: is_true + #### Count total rejections + #----------------------------------------------------------------------------- + - filter: Variable Assignment # compute sum 2*spikes+steps + assignments: + - name: DerivedMetaData/salinity_rejections + type: int + function: + name: IntObsFunction/LinearCombination + options: + variables: [DerivedMetaData/salinity_spikes, DerivedMetaData/salinity_steps] + coefs: [2,1] + #### Reject entire profile if total rejctions > threshold + #----------------------------------------------------------------------------- + - filter: Perform Action # reject whole profile if 2*spikes+steps>=rejection threshold + where: + - variable: + name: DerivedMetaData/salinity_rejections + minvalue: 3 + action: + name: reject + #------------------------------------------------------------------------------- + ### End of Spike test + #----------------------------------------------------------------------------- + + #----------------------------------------------------------------------------- + ## Filters on the whole profile: + #----------------------------------------------------------------------------- + #------------------------------------------------------------------------------- + ### Ocean Vertical stability test + #----------------------------------------------------------------------------- + + #### get pressure from depth + #----------------------------------------------------------------------------- + - filter: Variable Transforms + Transform: OceanDepthToPressure + ocean depth variable: depth + ocean depth group: MetaData + #### Create diagonostic flags for spike step + #----------------------------------------------------------------------------- + - filter: Create Diagnostic Flags + filter variables: + - name: DerivedObsValue/waterPressure + flags: + - name: DensitySpike + initial value: false + - name: DensityStep + initial value: false + - name: Superadiabat + initial value: false + #### + #----------------------------------------------------------------------------- + - filter: Ocean Vertical Stability Check + where: + - variable: + name: ObsValue/waterTemperature + value: is_valid + filter variables: + - name: DerivedObsValue/waterPressure # density spikes/steps --> flag d + variables: + temperature: ObsValue/waterTemperature + salinity: ObsValue/salinity + pressure: DerivedObsValue/waterPressure + count spikes: true + count steps: true + nominal tolerance: -0.05 + threshold: 0.25 + actions: + - name: set + flag: Superadiabat + - name: reject + #### where there are any density inversions, reject temperature only: + #----------------------------------------------------------------------------- + - filter: Perform Action + filter variables: + - name: ObsValue/waterTemperature + where: + - variable: + name: DiagnosticFlags/Superadiabat/waterPressure + value: is_true + action: + name: reject + #### where density spikes, reject all vars (temperature and salinity): + #----------------------------------------------------------------------------- + - filter: Perform Action + filter variables: + - name: ObsValue/waterTemperature + - name: ObsValue/salinity + where: + - variable: + name: DiagnosticFlags/DensitySpike/waterPressure + value: is_true + action: + name: reject + #### create derived metadata counting levels: + #----------------------------------------------------------------------------- + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/number_of_levels + type: int + function: + name: IntObsFunction/ProfileLevelCount + options: + where: + - variable: + name: ObsValue/waterTemperature + value: is_valid + #### create derived metadata counting spikes and steps: + #----------------------------------------------------------------------------- + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ocean_density_inversions + type: int + function: + name: IntObsFunction/ProfileLevelCount + options: + where: + - variable: + name: DiagnosticFlags/DensitySpike/waterPressure + value: is_true + - variable: + name: DiagnosticFlags/DensityStep/waterPressure + value: is_true + where operator: or + + #### whole profile is rejected if spikes+steps >= numlev/4, so compute + #### 4*( sum spikes+steps ) minus numlev + #### in order to check it against 0: + #----------------------------------------------------------------------------- + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ocean_density_rejections + type: int + function: + name: IntObsFunction/LinearCombination + options: + variables: [DerivedMetaData/ocean_density_inversions, DerivedMetaData/number_of_levels] + coefs: [4, -1] + #### reject whole profile if spikes+steps >= numlev/4 AND >= 2: + #----------------------------------------------------------------------------- + - filter: Perform Action + filter variables: + - name: ObsValue/waterTemperature + - name: ObsValue/salinity + where: + - variable: + name: DerivedMetaData/ocean_density_rejections + minvalue: 0 + - variable: + name: DerivedMetaData/ocean_density_inversions + minvalue: 2 + where operator: and + action: + name: reject + #------------------------------------------------------------------------------- + ### End of ocean vertical stability test + #----------------------------------------------------------------------------- + + #----------------------------------------------------------------------------- + ### If T was bad, remove S as well regardless + #----------------------------------------------------------------------------- + - filter: RejectList + where: + - variable: QCflagsData/waterTemperature + minvalue: 1 + defer to post: true diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 new file mode 100644 index 000000000..8223f3617 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 @@ -0,0 +1,17 @@ +- obs space: + name: insitu_profile_bathy + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [waterTemperature] + io pool: + max pool size: 1 + obs operator: + name: InsituTemperature + obs error: + covariance model: diagonal diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_dbuoy.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_dbuoy.yaml.j2 new file mode 100644 index 000000000..cec36c123 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/insitu_profile_dbuoy.yaml.j2 @@ -0,0 +1,22 @@ +- obs space: + name: insitu_profile_dbuoy + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [waterTemperature] + observed variables: [waterTemperature] + io pool: + max pool size: 1 + obs operator: + name: Composite + components: + - name: InsituTemperature + variables: + - name: waterTemperature + obs error: + covariance model: diagonal diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_dbuoyb.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_dbuoyb.yaml.j2 new file mode 100644 index 000000000..5cec54d67 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/insitu_profile_dbuoyb.yaml.j2 @@ -0,0 +1,22 @@ +- obs space: + name: insitu_profile_dbuoyb + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [waterTemperature] + observed variables: [waterTemperature] + io pool: + max pool size: 1 + obs operator: + name: Composite + components: + - name: InsituTemperature + variables: + - name: waterTemperature + obs error: + covariance model: diagonal diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_glider.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_glider.yaml.j2 new file mode 100644 index 000000000..1c9bab393 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/insitu_profile_glider.yaml.j2 @@ -0,0 +1,22 @@ +- obs space: + name: insitu_profile_glider + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [waterTemperature] + observed variables: [waterTemperature] + io pool: + max pool size: 1 + obs operator: + name: Composite + components: + - name: InsituTemperature + variables: + - name: waterTemperature + obs error: + covariance model: diagonal diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_marinemammal.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_marinemammal.yaml.j2 new file mode 100644 index 000000000..ad4456565 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/insitu_profile_marinemammal.yaml.j2 @@ -0,0 +1,22 @@ +- obs space: + name: insitu_profile_marinemammal + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [waterTemperature] + observed variables: [waterTemperature] + io pool: + max pool size: 1 + obs operator: + name: Composite + components: + - name: InsituTemperature + variables: + - name: waterTemperature + obs error: + covariance model: diagonal diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_mbuoy.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_mbuoy.yaml.j2 new file mode 100644 index 000000000..bf783ca76 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/insitu_profile_mbuoy.yaml.j2 @@ -0,0 +1,22 @@ +- obs space: + name: insitu_profile_mbuoy + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [waterTemperature] + observed variables: [waterTemperature] + io pool: + max pool size: 1 + obs operator: + name: Composite + components: + - name: InsituTemperature + variables: + - name: waterTemperature + obs error: + covariance model: diagonal diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_mbuoyb.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_mbuoyb.yaml.j2 new file mode 100644 index 000000000..78c261734 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/insitu_profile_mbuoyb.yaml.j2 @@ -0,0 +1,22 @@ +- obs space: + name: insitu_profile_mbuoyb + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [waterTemperature] + observed variables: [waterTemperature] + io pool: + max pool size: 1 + obs operator: + name: Composite + components: + - name: InsituTemperature + variables: + - name: waterTemperature + obs error: + covariance model: diagonal diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_tesac.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_tesac.yaml.j2 new file mode 100644 index 000000000..0c9af5751 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/insitu_profile_tesac.yaml.j2 @@ -0,0 +1,20 @@ +- obs space: + name: insitu_profile_tesac + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [waterTemperature] + observed variables: [waterTemperature] + io pool: + max pool size: 1 + obs operator: + name: Composite + components: + - name: InsituTemperature + variables: + - name: waterTemperature diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_tesac_salinity.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_tesac_salinity.yaml.j2 new file mode 100644 index 000000000..36c50035e --- /dev/null +++ b/parm/jcb-gdas/observations/marine/insitu_profile_tesac_salinity.yaml.j2 @@ -0,0 +1,22 @@ +- obs space: + name: insitu_profile_tesac_salinity + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [salinity] + observed variables: [salinity] + io pool: + max pool size: 1 + obs operator: + name: VertInterp + observation alias file: ./obsop_name_map.yaml + vertical coordinate: sea_water_depth + observation vertical coordinate: depth + interpolation method: linear + obs error: + covariance model: diagonal diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_xbtctd.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_xbtctd.yaml.j2 new file mode 100644 index 000000000..aee91cf79 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/insitu_profile_xbtctd.yaml.j2 @@ -0,0 +1,22 @@ +- obs space: + name: insitu_profile_xbtctd + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [waterTemperature] + observed variables: [waterTemperature] + io pool: + max pool size: 1 + obs operator: + name: Composite + components: + - name: InsituTemperature + variables: + - name: waterTemperature + obs error: + covariance model: diagonal diff --git a/parm/jcb-gdas/observations/marine/insitu_surface_altkob.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_surface_altkob.yaml.j2 new file mode 100644 index 000000000..063c3ec51 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/insitu_surface_altkob.yaml.j2 @@ -0,0 +1,18 @@ +- obs space: + name: insitu_surface_altkob + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaSurfaceTemperature] + io pool: + max pool size: 1 + obs operator: + name: Identity + observation alias file: obsop_name_map.yaml + obs error: + covariance model: diagonal diff --git a/parm/jcb-gdas/observations/marine/insitu_surface_trkob.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_surface_trkob.yaml.j2 new file mode 100644 index 000000000..625129273 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/insitu_surface_trkob.yaml.j2 @@ -0,0 +1,18 @@ +- obs space: + name: insitu_surface_trkob + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaSurfaceTemperature] + io pool: + max pool size: 1 + obs operator: + name: Identity + observation alias file: ./obsop_name_map.yaml + obs error: + covariance model: diagonal diff --git a/parm/jcb-gdas/observations/marine/insitu_surface_trkob_salinity.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_surface_trkob_salinity.yaml.j2 new file mode 100644 index 000000000..5ba67e942 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/insitu_surface_trkob_salinity.yaml.j2 @@ -0,0 +1,18 @@ +- obs space: + name: insitu_surface_trkob_salinity + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaSurfaceSalinity] + get values: + time interpolation: linear + obs operator: + name: Identity + observation alias file: ./obsop_name_map.yaml + obs error: + covariance model: diagonal diff --git a/parm/jcb-gdas/observations/marine/sst_abi_g16_l3c.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_abi_g16_l3c.yaml.j2 new file mode 100644 index 000000000..184fc29ee --- /dev/null +++ b/parm/jcb-gdas/observations/marine/sst_abi_g16_l3c.yaml.j2 @@ -0,0 +1,56 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaSurfaceTemperature] + io pool: + max pool size: 1 + get values: + time interpolation: linear + obs operator: + name: Identity + observation alias file: obsop_name_map.yaml + obs error: + covariance model: diagonal + obs filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + minvalue: 0.9 + - filter: Bounds Check + minvalue: 1.0 + maxvalue: 41.0 + - filter: Background Check + absolute threshold: 5.0 + - filter: Domain Check + where: + - variable: {name: ObsError/seaSurfaceTemperature} + minvalue: 1.0e-8 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_ice_area_fraction} + maxvalue: 1.0e-5 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_surface_temperature} + minvalue: -1.0 + #- filter: Domain Check + # where: + # - variable: {name: GeoVaLs/distance_from_coast} + # minvalue: 100e3 + - filter: Perform Action + action: + name: assign error + error function: + name: ObsFunction/LinearCombination + options: + variables: + - ObsError/seaSurfaceTemperature + coefs: + - 0.05 diff --git a/parm/jcb-gdas/observations/marine/sst_abi_g17_l3c.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_abi_g17_l3c.yaml.j2 new file mode 100644 index 000000000..184fc29ee --- /dev/null +++ b/parm/jcb-gdas/observations/marine/sst_abi_g17_l3c.yaml.j2 @@ -0,0 +1,56 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaSurfaceTemperature] + io pool: + max pool size: 1 + get values: + time interpolation: linear + obs operator: + name: Identity + observation alias file: obsop_name_map.yaml + obs error: + covariance model: diagonal + obs filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + minvalue: 0.9 + - filter: Bounds Check + minvalue: 1.0 + maxvalue: 41.0 + - filter: Background Check + absolute threshold: 5.0 + - filter: Domain Check + where: + - variable: {name: ObsError/seaSurfaceTemperature} + minvalue: 1.0e-8 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_ice_area_fraction} + maxvalue: 1.0e-5 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_surface_temperature} + minvalue: -1.0 + #- filter: Domain Check + # where: + # - variable: {name: GeoVaLs/distance_from_coast} + # minvalue: 100e3 + - filter: Perform Action + action: + name: assign error + error function: + name: ObsFunction/LinearCombination + options: + variables: + - ObsError/seaSurfaceTemperature + coefs: + - 0.05 diff --git a/parm/jcb-gdas/observations/marine/sst_ahi_h08_l3c.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_ahi_h08_l3c.yaml.j2 new file mode 100644 index 000000000..184fc29ee --- /dev/null +++ b/parm/jcb-gdas/observations/marine/sst_ahi_h08_l3c.yaml.j2 @@ -0,0 +1,56 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaSurfaceTemperature] + io pool: + max pool size: 1 + get values: + time interpolation: linear + obs operator: + name: Identity + observation alias file: obsop_name_map.yaml + obs error: + covariance model: diagonal + obs filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + minvalue: 0.9 + - filter: Bounds Check + minvalue: 1.0 + maxvalue: 41.0 + - filter: Background Check + absolute threshold: 5.0 + - filter: Domain Check + where: + - variable: {name: ObsError/seaSurfaceTemperature} + minvalue: 1.0e-8 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_ice_area_fraction} + maxvalue: 1.0e-5 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_surface_temperature} + minvalue: -1.0 + #- filter: Domain Check + # where: + # - variable: {name: GeoVaLs/distance_from_coast} + # minvalue: 100e3 + - filter: Perform Action + action: + name: assign error + error function: + name: ObsFunction/LinearCombination + options: + variables: + - ObsError/seaSurfaceTemperature + coefs: + - 0.05 diff --git a/parm/jcb-gdas/observations/marine/sst_avhrr_ma_l3u.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_avhrr_ma_l3u.yaml.j2 new file mode 100644 index 000000000..184fc29ee --- /dev/null +++ b/parm/jcb-gdas/observations/marine/sst_avhrr_ma_l3u.yaml.j2 @@ -0,0 +1,56 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaSurfaceTemperature] + io pool: + max pool size: 1 + get values: + time interpolation: linear + obs operator: + name: Identity + observation alias file: obsop_name_map.yaml + obs error: + covariance model: diagonal + obs filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + minvalue: 0.9 + - filter: Bounds Check + minvalue: 1.0 + maxvalue: 41.0 + - filter: Background Check + absolute threshold: 5.0 + - filter: Domain Check + where: + - variable: {name: ObsError/seaSurfaceTemperature} + minvalue: 1.0e-8 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_ice_area_fraction} + maxvalue: 1.0e-5 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_surface_temperature} + minvalue: -1.0 + #- filter: Domain Check + # where: + # - variable: {name: GeoVaLs/distance_from_coast} + # minvalue: 100e3 + - filter: Perform Action + action: + name: assign error + error function: + name: ObsFunction/LinearCombination + options: + variables: + - ObsError/seaSurfaceTemperature + coefs: + - 0.05 diff --git a/parm/jcb-gdas/observations/marine/sst_avhrr_mb_l3u.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_avhrr_mb_l3u.yaml.j2 new file mode 100644 index 000000000..184fc29ee --- /dev/null +++ b/parm/jcb-gdas/observations/marine/sst_avhrr_mb_l3u.yaml.j2 @@ -0,0 +1,56 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaSurfaceTemperature] + io pool: + max pool size: 1 + get values: + time interpolation: linear + obs operator: + name: Identity + observation alias file: obsop_name_map.yaml + obs error: + covariance model: diagonal + obs filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + minvalue: 0.9 + - filter: Bounds Check + minvalue: 1.0 + maxvalue: 41.0 + - filter: Background Check + absolute threshold: 5.0 + - filter: Domain Check + where: + - variable: {name: ObsError/seaSurfaceTemperature} + minvalue: 1.0e-8 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_ice_area_fraction} + maxvalue: 1.0e-5 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_surface_temperature} + minvalue: -1.0 + #- filter: Domain Check + # where: + # - variable: {name: GeoVaLs/distance_from_coast} + # minvalue: 100e3 + - filter: Perform Action + action: + name: assign error + error function: + name: ObsFunction/LinearCombination + options: + variables: + - ObsError/seaSurfaceTemperature + coefs: + - 0.05 diff --git a/parm/jcb-gdas/observations/marine/sst_avhrr_mc_l3u.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_avhrr_mc_l3u.yaml.j2 new file mode 100644 index 000000000..184fc29ee --- /dev/null +++ b/parm/jcb-gdas/observations/marine/sst_avhrr_mc_l3u.yaml.j2 @@ -0,0 +1,56 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaSurfaceTemperature] + io pool: + max pool size: 1 + get values: + time interpolation: linear + obs operator: + name: Identity + observation alias file: obsop_name_map.yaml + obs error: + covariance model: diagonal + obs filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + minvalue: 0.9 + - filter: Bounds Check + minvalue: 1.0 + maxvalue: 41.0 + - filter: Background Check + absolute threshold: 5.0 + - filter: Domain Check + where: + - variable: {name: ObsError/seaSurfaceTemperature} + minvalue: 1.0e-8 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_ice_area_fraction} + maxvalue: 1.0e-5 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_surface_temperature} + minvalue: -1.0 + #- filter: Domain Check + # where: + # - variable: {name: GeoVaLs/distance_from_coast} + # minvalue: 100e3 + - filter: Perform Action + action: + name: assign error + error function: + name: ObsFunction/LinearCombination + options: + variables: + - ObsError/seaSurfaceTemperature + coefs: + - 0.05 diff --git a/parm/jcb-gdas/observations/marine/sst_viirs_n20_l3u.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_viirs_n20_l3u.yaml.j2 new file mode 100644 index 000000000..184fc29ee --- /dev/null +++ b/parm/jcb-gdas/observations/marine/sst_viirs_n20_l3u.yaml.j2 @@ -0,0 +1,56 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaSurfaceTemperature] + io pool: + max pool size: 1 + get values: + time interpolation: linear + obs operator: + name: Identity + observation alias file: obsop_name_map.yaml + obs error: + covariance model: diagonal + obs filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + minvalue: 0.9 + - filter: Bounds Check + minvalue: 1.0 + maxvalue: 41.0 + - filter: Background Check + absolute threshold: 5.0 + - filter: Domain Check + where: + - variable: {name: ObsError/seaSurfaceTemperature} + minvalue: 1.0e-8 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_ice_area_fraction} + maxvalue: 1.0e-5 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_surface_temperature} + minvalue: -1.0 + #- filter: Domain Check + # where: + # - variable: {name: GeoVaLs/distance_from_coast} + # minvalue: 100e3 + - filter: Perform Action + action: + name: assign error + error function: + name: ObsFunction/LinearCombination + options: + variables: + - ObsError/seaSurfaceTemperature + coefs: + - 0.05 diff --git a/parm/jcb-gdas/observations/marine/sst_viirs_npp_l3u.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_viirs_npp_l3u.yaml.j2 new file mode 100644 index 000000000..184fc29ee --- /dev/null +++ b/parm/jcb-gdas/observations/marine/sst_viirs_npp_l3u.yaml.j2 @@ -0,0 +1,56 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaSurfaceTemperature] + io pool: + max pool size: 1 + get values: + time interpolation: linear + obs operator: + name: Identity + observation alias file: obsop_name_map.yaml + obs error: + covariance model: diagonal + obs filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + minvalue: 0.9 + - filter: Bounds Check + minvalue: 1.0 + maxvalue: 41.0 + - filter: Background Check + absolute threshold: 5.0 + - filter: Domain Check + where: + - variable: {name: ObsError/seaSurfaceTemperature} + minvalue: 1.0e-8 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_ice_area_fraction} + maxvalue: 1.0e-5 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_surface_temperature} + minvalue: -1.0 + #- filter: Domain Check + # where: + # - variable: {name: GeoVaLs/distance_from_coast} + # minvalue: 100e3 + - filter: Perform Action + action: + name: assign error + error function: + name: ObsFunction/LinearCombination + options: + variables: + - ObsError/seaSurfaceTemperature + coefs: + - 0.05 From d4da61f9d22098f39796cc8d2c45283c8befc97b Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Tue, 30 Jul 2024 11:15:32 -0400 Subject: [PATCH 043/199] update to snow recentering --- .../snow/fv3jedi_land_ensrecenter.yaml.j2 | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/parm/jcb-gdas/algorithm/snow/fv3jedi_land_ensrecenter.yaml.j2 b/parm/jcb-gdas/algorithm/snow/fv3jedi_land_ensrecenter.yaml.j2 index f7236a8c0..48ea543ca 100644 --- a/parm/jcb-gdas/algorithm/snow/fv3jedi_land_ensrecenter.yaml.j2 +++ b/parm/jcb-gdas/algorithm/snow/fv3jedi_land_ensrecenter.yaml.j2 @@ -14,7 +14,20 @@ date: "{{ snow_background_time_iso }}" variables: [totalSnowDepth] increment mask: variable: [slmsk] - maxvalue: 0.99 + minvalue: 0.99 + maxvalue: 1.01 + background: + datetime: "{{ snow_background_time_iso }}" + filetype: fms restart + state variables: [snodl,vtype,slmsk] + datapath: ./bkg/mem001 + filename is datetime templated: true + filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" + filename_trcr: "%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc" + filename_sfcd: "%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc" + filename_sfcw: "%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc" + filename_cplr: "%yyyy%mm%dd.%hh%MM%ss.coupler.res" + skip coupler file: true deterministic increment: datetime: "{{ snow_increment_time_iso }}" filetype: fms restart From d24ca1ae347a519556f489ee1d8ebaae91cdddf9 Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Tue, 30 Jul 2024 11:20:51 -0400 Subject: [PATCH 044/199] reduce variables in snow recenter template --- .../algorithm/snow/fv3jedi_land_ensrecenter.yaml.j2 | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/parm/jcb-gdas/algorithm/snow/fv3jedi_land_ensrecenter.yaml.j2 b/parm/jcb-gdas/algorithm/snow/fv3jedi_land_ensrecenter.yaml.j2 index 48ea543ca..81d6f9ce7 100644 --- a/parm/jcb-gdas/algorithm/snow/fv3jedi_land_ensrecenter.yaml.j2 +++ b/parm/jcb-gdas/algorithm/snow/fv3jedi_land_ensrecenter.yaml.j2 @@ -19,7 +19,7 @@ increment mask: background: datetime: "{{ snow_background_time_iso }}" filetype: fms restart - state variables: [snodl,vtype,slmsk] + state variables: [slmsk] datapath: ./bkg/mem001 filename is datetime templated: true filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" @@ -31,7 +31,7 @@ increment mask: deterministic increment: datetime: "{{ snow_increment_time_iso }}" filetype: fms restart - state variables: [snodl,vtype,slmsk] + state variables: [snodl] datapath: ./inc/det_ensres filename is datetime templated: true filename_core: "snowinc.%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" @@ -43,7 +43,7 @@ deterministic increment: deterministic background: datetime: "{{ snow_background_time_iso }}" filetype: fms restart - state variables: [snodl,vtype,slmsk] + state variables: [snodl] datapath: ./bkg/det_ensres filename is datetime templated: true filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" @@ -70,7 +70,7 @@ ensemble backgrounds: template: datetime: "{{ snow_background_time_iso }}" filetype: fms restart - state variables: [snodl,vtype,slmsk] + state variables: [snodl] datapath: ./bkg/mem%mem% filename is datetime templated: true filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" From 7b3b8c6d34732bcdd05390c3f700934dc4368300 Mon Sep 17 00:00:00 2001 From: DavidNew-NOAA <134300700+DavidNew-NOAA@users.noreply.github.com> Date: Tue, 6 Aug 2024 13:34:22 -0400 Subject: [PATCH 045/199] Stage atmospheric backgrounds and UFS cubed-sphere history files (#19) This PR changes the G-W so that backgrounds for the atmospheric analyses are staged as UFS cubed-sphere history files rather than FMS restarts. The motivation being that FMS restarts include many data we don't need for the analyses and thus take up too much disk space/bandwidth. This is a companion PR to Global Workflow [#2791](https://github.com/NOAA-EMC/global-workflow/pull/2792) and GDASApp [#1236](https://github.com/NOAA-EMC/GDASApp/pull/1236). --- .../atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 | 18 ++++++++---------- .../fv3jedi_fv3inc_variational.yaml.j2 | 16 +++++++--------- .../atmosphere_3dvar_outer_loop_1.yaml.j2 | 2 +- .../atmosphere_3dvar_outer_loop_2.yaml.j2 | 2 +- .../atmosphere/atmosphere_background.yaml.j2 | 18 ++++++++---------- .../atmosphere_background_ensemble.yaml.j2 | 17 ++++++++--------- ...background_error_hybrid_gsibec_bump.yaml.j2 | 11 ++++------- .../atmosphere_geometry_background.yaml.j2 | 2 +- 8 files changed, 38 insertions(+), 48 deletions(-) diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 index c7d24cacf..f69bca233 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 @@ -1,6 +1,6 @@ variable change: variable change name: Model2GeoVaLs - input variables: &bkgvars [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr,phis] + input variables: &bkgvars [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr,surface_geopotential_height] output variables: &fv3incrvars [ua,va,t,sphum,ice_wat,liq_wat,o3mr,delp,hydrostatic_delz] jedi increment variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] fv3 increment variables: *fv3incrvars @@ -15,7 +15,7 @@ background geometry: npx: {{ atmosphere_npx_ges }} npy: {{ atmosphere_npy_ges }} npz: {{ atmosphere_npz_ges }} - field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_restart.yaml + field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_history.yaml jedi increment geometry: fms initialization: namelist filename: ./fv3jedi/fmsmpp.nml @@ -27,7 +27,7 @@ jedi increment geometry: npx: {{ atmosphere_npx_ges }} npy: {{ atmosphere_npy_ges }} npz: {{ atmosphere_npz_ges }} - field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_restart.yaml + field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_history.yaml fv3 increment geometry: fms initialization: namelist filename: ./fv3jedi/fmsmpp.nml @@ -44,14 +44,12 @@ members from template: template: background input: datapath: ./bkg/mem%mem% - filetype: fms restart + filetype: cube sphere history + provider: ufs + ufs soil nlev: 4 + ufs fields split by level: [smc,stc] + filenames: [ {{ atmosphere_ensemble_history_prefix }}cubed_sphere_grid_atmf006.nc, {{ atmosphere_ensemble_history_prefix }}cubed_sphere_grid_sfcf006.nc ] datetime: '{{ atmosphere_background_time_iso }}' - filename is datetime templated: true - filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" - filename_trcr: "%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc" - filename_sfcd: "%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc" - filename_sfcw: "%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc" - filename_cplr: "%yyyy%mm%dd.%hh%MM%ss.coupler.res" state variables: *bkgvars jedi increment input: filetype: cube sphere history diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 index 63079a2c3..d5277f129 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 @@ -1,6 +1,6 @@ variable change: variable change name: Model2GeoVaLs - input variables: &bkgvars [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr,phis] + input variables: &bkgvars [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr,surface_geopotential_height] output variables: &fv3incrvars [ua,va,t,sphum,ice_wat,liq_wat,o3mr,delp,hydrostatic_delz] jedi increment variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] fv3 increment variables: *fv3incrvars @@ -15,7 +15,7 @@ background geometry: npx: {{ atmosphere_npx_ges }} npy: {{ atmosphere_npy_ges }} npz: {{ atmosphere_npz_ges }} - field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_restart.yaml + field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_history.yaml jedi increment geometry: fms initialization: namelist filename: ./fv3jedi/fmsmpp.nml @@ -43,14 +43,12 @@ fv3 increment geometry: members: - background input: datapath: ./bkg - filetype: fms restart + filetype: cube sphere history + provider: ufs + ufs soil nlev: 4 + ufs fields split by level: [smc,stc] datetime: "{{ atmosphere_background_time_iso }}" - filename is datetime templated: true - filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" - filename_trcr: "%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc" - filename_sfcd: "%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc" - filename_sfcw: "%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc" - filename_cplr: "%yyyy%mm%dd.%hh%MM%ss.coupler.res" + filenames: [ {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_atmf006.nc, {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_sfcf006.nc ] state variables: *bkgvars jedi increment input: filetype: cube sphere history diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 index 9c850f633..3b65a686c 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 @@ -12,6 +12,6 @@ npx: {{atmosphere_npx_anl}} npy: {{atmosphere_npy_anl}} npz: {{atmosphere_npz_anl}} - field metadata override: {{atmosphere_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml + field metadata override: {{atmosphere_fv3jedi_files_path}}/fv3jedi_fieldmetadata_history.yaml diagnostics: departures: bkgmob diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 index ec5dc0950..5ca7f8082 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 @@ -12,6 +12,6 @@ npx: {{atmosphere_npx_anl}} npy: {{atmosphere_npy_anl}} npz: {{atmosphere_npz_anl}} - field metadata override: {{atmosphere_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml + field metadata override: {{atmosphere_fv3jedi_files_path}}/fv3jedi_fieldmetadata_history.yaml diagnostics: departures: bkgmob1 diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 index e5a882731..580fc68ac 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 @@ -1,12 +1,10 @@ -datapath: {{ atmosphere_background_path}} -filetype: fms restart +datapath: {{ atmosphere_background_path }} +filetype: cube sphere history +provider: ufs +ufs soil nlev: 4 +ufs fields split by level: [smc,stc] datetime: "{{ atmosphere_background_time_iso }}" -filename is datetime templated: true -filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" -filename_trcr: "%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc" -filename_sfcd: "%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc" -filename_sfcw: "%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc" -filename_cplr: "%yyyy%mm%dd.%hh%MM%ss.coupler.res" -state variables: [ua,va,t,delp,ps,sphum,ice_wat,liq_wat,o3mr,phis, - slmsk,sheleg,tsea,vtype,stype,vfrac,stc,smc,snwdph, +filenames: [ {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_atmf006.nc, {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_sfcf006.nc ] +state variables: [ua,va,t,delp,ps,sphum,ice_wat,liq_wat,o3mr,surface_geopotential_height, + slmsk,sheleg,tsea,vtype,stype,vfrac,stc,smc,snwdphMeters, u_srf,v_srf,f10m] diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_background_ensemble.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background_ensemble.yaml.j2 index 37640ee33..16b980e40 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_background_ensemble.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background_ensemble.yaml.j2 @@ -2,17 +2,16 @@ datetime: '{{ atmosphere_background_time_iso }}' members from template: template: datetime: '{{ atmosphere_background_time_iso }}' - filetype: fms restart - state variables: [ua,va,t,DZ,delp,ps,sphum,ice_wat,liq_wat,o3mr,phis, - slmsk,sheleg,tsea,vtype,stype,vfrac,stc,smc,snwdph, + filetype: cube sphere history + provider: ufs + ufs soil nlev: 4 + ufs fields split by level: [smc,stc] + state variables: [ua,va,t,delz,delp,ps,sphum,ice_wat,liq_wat,o3mr,surface_geopotential_height, + slmsk,sheleg,tsea,vtype,stype,vfrac,stc,smc,snwdphMeters, u_srf,v_srf,f10m] - datapath: {{ atmosphere_background_ensemble_path}} + datapath: {{ atmosphere_background_ensemble_path }} filename is datetime templated: true - filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" - filename_trcr: "%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc" - filename_sfcd: "%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc" - filename_sfcw: "%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc" - filename_cplr: "%yyyy%mm%dd.%hh%MM%ss.coupler.res" + filenames: [ {{ atmosphere_ensemble_history_prefix }}cubed_sphere_grid_atmf006.nc, {{ atmosphere_ensemble_history_prefix }}cubed_sphere_grid_sfcf006.nc ] pattern: "{{ atmosphere_ensemble_pattern | default("%mem%", true) }}" nmembers: {{ atmosphere_number_ensemble_members }} zero padding: 3 diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 index 4ba0dbe48..e5b06aa69 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 @@ -32,15 +32,12 @@ components: members from template: template: datetime: "{{ atmosphere_background_time_iso }}" - filetype: fms restart + filetype: cube sphere history + provider: ufs state variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] - datapath: {{ atmosphere_background_ensemble_path}} + datapath: {{ atmosphere_background_ensemble_path }} filename is datetime templated: true - filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" - filename_trcr: "%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc" - filename_sfcd: "%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc" - filename_sfcw: "%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc" - filename_cplr: "%yyyy%mm%dd.%hh%MM%ss.coupler.res" + filenames: [ {{ atmosphere_ensemble_history_prefix }}cubed_sphere_grid_atmf006.nc, {{ atmosphere_ensemble_history_prefix }}cubed_sphere_grid_sfcf006.nc ] pattern: "{{ atmosphere_ensemble_pattern | default("%mem%", true) }}" nmembers: {{atmosphere_number_ensemble_members}} zero padding: 3 diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_geometry_background.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_geometry_background.yaml.j2 index 65ec71bbf..8e0f501ea 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_geometry_background.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_geometry_background.yaml.j2 @@ -8,4 +8,4 @@ layout: npx: {{atmosphere_npx_ges}} npy: {{atmosphere_npy_ges}} npz: {{atmosphere_npz_ges}} -field metadata override: "{{atmosphere_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" +field metadata override: "{{atmosphere_fv3jedi_files_path}}/fv3jedi_fieldmetadata_history.yaml" From 82751e9aacc03eb507c1ef968688c8ae0d232550 Mon Sep 17 00:00:00 2001 From: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com> Date: Thu, 8 Aug 2024 16:49:43 -0400 Subject: [PATCH 046/199] Update observation yamls for atmosphere var and local ensemble DA (#21) This PR updates observation templates use by the variational and local ensemble atmosphere applications. The following changes are included in this PR - updating radiance bias correction keywords - renaming templates to match bufr2ioda converter names - adding observation templates for lgetkf Resolves #20 --- ...{iasi_metop-a.yaml => mtiasi_metop-a.yaml} | 0 ...{iasi_metop-b.yaml => mtiasi_metop-b.yaml} | 0 .../atmosphere-lgetkf/aircraft.yaml.j2 | 365 +++++++++++ .../atmosphere-lgetkf/atms_n20.yaml.j2 | 546 ++++++++++++++++ .../atmosphere-lgetkf/atms_npp.yaml.j2 | 564 ++++++++++++++++ .../atmosphere-lgetkf/conventional_ps.yaml.j2 | 373 +++++++++++ .../atmosphere-lgetkf/mtiasi_metop-a.yaml.j2 | 588 +++++++++++++++++ .../atmosphere-lgetkf/mtiasi_metop-b.yaml.j2 | 588 +++++++++++++++++ .../atmosphere-lgetkf/ompsnp_npp.yaml.j2 | 339 ++++++++++ .../atmosphere-lgetkf/ompstc_npp.yaml.j2 | 144 +++++ .../satwnd.abi_goes-18.yaml.j2 | 610 ++++++++++++++++++ .../satwnd.leogeo_multi.yaml.j2 | 270 ++++++++ .../satwnd.viirs_n20.yaml.j2 | 298 +++++++++ .../satwnd.viirs_npp.yaml.j2 | 298 +++++++++ .../observations/atmosphere/atms_n20.yaml.j2 | 14 +- .../observations/atmosphere/atms_npp.yaml.j2 | 14 +- ...onv_ps.yaml.j2 => conventional_ps.yaml.j2} | 0 .../atmosphere/cris-fsr_n20.yaml.j2 | 14 +- .../atmosphere/cris-fsr_npp.yaml.j2 | 14 +- ...metop-a.yaml.j2 => mtiasi_metop-a.yaml.j2} | 14 +- ...metop-b.yaml.j2 => mtiasi_metop-b.yaml.j2} | 14 +- .../atmosphere/satwnd.abi_goes-18.yaml.j2 | 601 +++++++++++++++++ .../observations/atmosphere/ssmis_f17.yaml.j2 | 14 +- .../observations/atmosphere/ssmis_f18.yaml.j2 | 14 +- 24 files changed, 5640 insertions(+), 56 deletions(-) rename parm/jcb-gdas/observation_chronicle/atmosphere/{iasi_metop-a.yaml => mtiasi_metop-a.yaml} (100%) rename parm/jcb-gdas/observation_chronicle/atmosphere/{iasi_metop-b.yaml => mtiasi_metop-b.yaml} (100%) create mode 100644 parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere-lgetkf/atms_n20.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere-lgetkf/atms_npp.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere-lgetkf/mtiasi_metop-a.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere-lgetkf/mtiasi_metop-b.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere-lgetkf/ompsnp_npp.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere-lgetkf/ompstc_npp.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-18.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.leogeo_multi.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.viirs_n20.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.viirs_npp.yaml.j2 rename parm/jcb-gdas/observations/atmosphere/{conv_ps.yaml.j2 => conventional_ps.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{iasi_metop-a.yaml.j2 => mtiasi_metop-a.yaml.j2} (99%) rename parm/jcb-gdas/observations/atmosphere/{iasi_metop-b.yaml.j2 => mtiasi_metop-b.yaml.j2} (99%) create mode 100644 parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-18.yaml.j2 diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/iasi_metop-a.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/mtiasi_metop-a.yaml similarity index 100% rename from parm/jcb-gdas/observation_chronicle/atmosphere/iasi_metop-a.yaml rename to parm/jcb-gdas/observation_chronicle/atmosphere/mtiasi_metop-a.yaml diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/iasi_metop-b.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/mtiasi_metop-b.yaml similarity index 100% rename from parm/jcb-gdas/observation_chronicle/atmosphere/iasi_metop-b.yaml rename to parm/jcb-gdas/observation_chronicle/atmosphere/mtiasi_metop-b.yaml diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml.j2 new file mode 100644 index 000000000..4d4a72ec7 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml.j2 @@ -0,0 +1,365 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: Aircraft + distribution: + name: Halo + halo size: 1250e3 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsgrouping: + group variables: ["stationIdentification"] + sort variable: "pressure" + sort order: "descending" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward, airTemperature, specificHumidity] + # + + # Observation Operator + # -------------------- + obs operator: + name: VertInterp + # + + # Observation Filters (QC) + # ------------------------ + obs filters: + # Observation Range Sanity Check + - filter: Bounds Check + filter variables: + - name: airTemperature + minvalue: 195 + maxvalue: 327 + action: + name: reject + # + - filter: Bounds Check + filter variables: + - name: specificHumidity + minvalue: 1.0E-7 + maxvalue: 0.034999999 + action: + name: reject + # + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130 + maxvalue: 130 + action: + name: reject + # + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130.0 + action: + name: reject + # Reject when pressure is less than 126 mb. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: MetaData/pressure + minvalue: 12600 + action: + name: reject + # + # Reject all obs with PreQC mark already set above 3 + # - filter: PreQC + # maxvalue: 3 + # action: + # name: reject + # + #-------------------------------------------------------------------------------------------------------------------- + # Wind + #-------------------------------------------------------------------------------------------------------------------- + # + # Begin by assigning all ObsError to a constant value. These will get overwritten (as needed) for specific types. + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error parameter: 2.0 # 2.0 m/s + # Assign intial ObsError specific to AIREP/ACARS + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error parameter: 3.6 # 3.6 m/s + where: + - variable: + name: ObsType/windEastward + is_in: 230 + # Assign intial ObsError specific to AMDAR + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error parameter: 3.0 # 3.0 m/s + where: + - variable: + name: ObsType/windEastward + is_in: 231 + # Assign intial ObsError specific to MDCRS + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error parameter: 2.5 # 2.5 m/s + where: + - variable: + name: ObsType/windEastward + is_in: 233 + # + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [100000, 95000, 80000, 65000, 60000, 55000, 50000, 45000, 40000, + 35000, 30000, 25000, 20000, 15000, 10000] #Pressure (Pa) + errors: [1.4, 1.5, 1.6, 1.8, 1.9, 2.0, 2.1, 2.3, 2.6, 2.8, 3.0, 3.2, 2.7, + 2.4, 2.1] + # Assign the initial ObsError, based on height/pressure for RECON aircraft + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [70000, 65000, 60000, 55000, 50000, 45000, 40000, 35000, 30000, 25000, + 20000, 15000, 10000, 7500, 5000] + errors: [2.4, 2.5, 2.6, 2.7, 2.8, 2.95, 3.1, 3.25, 3.4, 3.175, 2.95, 2.725, + 2.5, 2.6, 2.7] + where: + - variable: + name: ObsType/windEastward + is_in: 232 + # Reject when difference of wind direction is more than 50 degrees. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/WindDirAngleDiff + options: + minimum_uv: 3.5 + maxvalue: 50.0 + action: + name: reject + defer to post: true + # When multiple obs exist within a single vertical model level, inflate ObsError + # - filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsFunction/ObsErrorFactorConventional + # options: + # test QCflag: PreQC + # inflate variables: [windEastward] + # defer to post: true + # + # - filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsFunction/ObsErrorFactorConventional + # options: + # test QCflag: PreQC + # inflate variables: [windNorthward] + # defer to post: true + # + # Gross error check with (O - B) / ObsError greater than threshold. + - filter: Background Check + filter variables: + - name: windEastward + - name: windNorthward + threshold: 6.0 + absolute threshold: 19.0 + action: + name: reject + # + #-------------------------------------------------------------------------------------------------------------------- + # Temperature + #-------------------------------------------------------------------------------------------------------------------- + # + # Begin by assigning all ObsError to a constant value. These will get overwritten for specific types. + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: assign error + error parameter: 2.0 # 2.0 K + # Assign the initial observation error, based on pressure (for AIREP/ACARS; itype=130) + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [100000, 95000, 90000, 85000, 80000] + errors: [2.5, 2.3, 2.1, 1.9, 1.7] + where: + - variable: + name: ObsType/airTemperature + is_in: 130 + # Assign the initial observation error, based on pressure (for AMDAR and MDCRS; itype=131,133) + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [100000, 95000, 90000, 85000, 80000] + errors: [1.4706, 1.3529, 1.2353, 1.1176, 1.0] + where: + - variable: + name: ObsType/airTemperature + is_in: 131,133 + # Assign the initial observation error, based on pressure (for RECON aircraft; itype=132) + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [100000, 95000, 90000, 85000, 35000, 30000, 25000, 20000, 15000, + 10000, 7500, 5000, 4000, 3200, 2000, 1000] + errors: [1.2, 1.1, 0.9, 0.8, 0.8, 0.9, 1.2, 1.2, 1.0, 0.8, 0.8, 0.9, 0.95, + 1.0, 1.25, 1.5] + where: + - variable: + name: ObsType/airTemperature + is_in: 132 + # When multiple obs exist within a single vertical model level, inflate ObsError + # - filter: Perform Action + # filter variables: + # - name: airTemperature + # action: + # name: inflate error + # inflation variable: + # name: ObsFunction/ObsErrorFactorConventional + # options: + # test QCflag: PreQC + # inflate variables: [airTemperature] + # defer to post: true + # Gross error check with (O - B) / ObsError greater than threshold. + - filter: Background Check + filter variables: + - name: airTemperature + threshold: 7.0 + absolute threshold: 8.0 + action: + name: reject + # + #-------------------------------------------------------------------------------------------------------------------- + # Moisture + #-------------------------------------------------------------------------------------------------------------------- + # + # Assign the initial observation error, based on height/pressure ONLY MDCRS + - filter: Perform Action + filter variables: + - name: specificHumidity + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000, 105000, 100000, 95000, 90000, 85000, 80000, 75000, 70000, + 65000, 60000, 55000, 50000, 45000, 40000, 35000, 30000, 25000, 20000, + 15000, 10000, 7500, 5000, 4000, 3000] + errors: [.19455, .19062, .18488, .17877, .17342, .16976, .16777, .16696, + .16605, .16522, .16637, .17086, .17791, .18492, .18996, .19294, .19447, + .19597, .19748, .19866, .19941, .19979, .19994, .19999, .2] + scale_factor_var: ObsValue/specificHumidity + where: + - variable: + name: ObsType/specificHumidity + is_in: 133 + # When multiple obs exist within a single vertical model level, inflate ObsError + # - filter: Perform Action + # filter variables: + # - name: specificHumidity + # action: + # name: inflate error + # inflation variable: + # name: ObsFunction/ObsErrorFactorConventional + # options: + # test QCflag: PreQC + # inflate variables: [specificHumidity] + # defer to post: true + # Gross error check with (O - B) / ObsError greater than threshold. + - filter: Background Check + filter variables: + - name: specificHumidity + threshold: 8.0 + action: + name: reject + + # Observation Localizations (LocalEnsembleDA) + # ------------------------------------------- + obs localizations: + - localization method: Horizontal Gaspari-Cohn + lengthscale: 1250e3 + max nobs: 10000 + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/atms_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/atms_n20.yaml.j2 new file mode 100644 index 000000000..7bfa5e1c2 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/atms_n20.yaml.j2 @@ -0,0 +1,546 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: ATMS N20 + distribution: + name: Halo + halo size: 1250e3 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [brightnessTemperature] + channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} + + # Observation Operator + # -------------------- + obs operator: + name: CRTM + Absorbers: [H2O, O3, CO2] + Clouds: [Water, Ice] + Cloud_Fraction: 1.0 + Cloud_Seeding: true + obs options: + Sensor_ID: &{{observation_from_jcb}}_sensor_id atms_n20 + EndianType: little_endian + CoefficientPath: "{{crtm_coefficient_path}}" + linear obs operator: + Absorbers: [H2O, O3] + + # Observation Bias Correction (VarBC) + # ----------------------------------- + obs bias: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" + variational bc: + predictors: + - name: constant + - name: lapseRate + order: 2 + tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" + - name: lapseRate + tlapse: *{{observation_from_jcb}}_tlapse + - name: emissivityJacobian + - name: sensorScanAngle + order: 4 + - name: sensorScanAngle + order: 3 + - name: sensorScanAngle + order: 2 + - name: sensorScanAngle + covariance: + minimal required obs number: 20 + variance range: [1.0e-6, 10.0] + step size: 1.0e-4 + largest analysis variance: 10000.0 + prior: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiascovin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovin_suffix}}" + inflation: + ratio: 1.1 + ratio for small dataset: 2.0 + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" + + # Observation Pre Filters (QC) + # ---------------------------- + obs pre filters: + # Step 0-A: Create Diagnostic Flags + - filter: Create Diagnostic Flags + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + flags: + - name: ScanEdgeRemoval + initial value: false + force reinitialization: false + - name: Thinning + initial value: false + force reinitialization: false + - name: CLWRetrievalCheck + initial value: false + force reinitialization: false + - name: WindowChannelExtremeResidual + initial value: false + force reinitialization: false + - name: HydrometeorCheck + initial value: false + force reinitialization: false + - name: GrossCheck + initial value: false + force reinitialization: false + - name: InterChannelConsistency + initial value: false + force reinitialization: false + - name: UseflagCheck + initial value: false + force reinitialization: false + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Step 0-B: Calculate derived variables + # Calculate CLW retrieved from observation + - filter: Variable Assignment + assignments: + - name: CLWRetFromObs@DerivedMetaData + type: float + function: + name: CLWRetMW@ObsFunction + options: + clwret_ch238: 1 + clwret_ch314: 2 + clwret_types: [ObsValue] + # Calculate CLW retrieved from observation + - filter: Variable Assignment + assignments: + - name: CLWRetFromBkg@DerivedMetaData + type: float + function: + name: CLWRetMW@ObsFunction + options: + clwret_ch238: 1 + clwret_ch314: 2 + clwret_types: [HofX] + # Calculate symmetric retrieved CLW + - filter: Variable Assignment + assignments: + - name: CLWRetSymmetric@DerivedMetaData + type: float + value: 1000.0 + + - filter: Variable Assignment + where: + - variable: + name: CLWRetFromObs@DerivedMetaData + minvalue: 0. + maxvalue: 999. + - variable: + name: CLWRetFromBkg@DerivedMetaData + minvalue: 0. + maxvalue: 999. + where operator: and + assignments: + - name: CLWRetSymmetric@DerivedMetaData + type: float + function: + name: Arithmetic@ObsFunction + options: + variables: + - name: CLWRetFromObs@DerivedMetaData + - name: CLWRetFromBkg@DerivedMetaData + total coefficient: 0.5 + + # Calculate scattering index from observation + - filter: Variable Assignment + assignments: + - name: SIRetFromObs@DerivedMetaData + type: float + function: + name: SCATRetMW@ObsFunction + options: + scatret_ch238: 1 + scatret_ch314: 2 + scatret_ch890: 16 + scatret_types: [ObsValue] + # Calculate CLW obs/bkg match index + - filter: Variable Assignment + assignments: + - name: CLWMatchIndex@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: CLWMatchIndexMW@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + clwobs_function: + name: CLWRetFromObs@DerivedMetaData + clwbkg_function: + name: CLWRetFromBkg@DerivedMetaData + clwret_clearsky: [0.030, 0.030, 0.030, 0.020, 0.030, 0.080, 0.150, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.020, 0.030, 0.030, + 0.030, 0.030, 0.050, 0.100] + # Calculate symmetric observation error + - filter: Variable Assignment + assignments: + - name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorModelRamp@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + xvar: + name: CLWRetSymmetric@DerivedMetaData + x0: [0.030, 0.030, 0.030, 0.020, 0.030, 0.080, 0.150, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.000, 0.020, 0.030, 0.030, 0.030, 0.030, + 0.050, 0.100] + x1: [0.350, 0.380, 0.400, 0.450, 0.500, 1.000, 1.000, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.000, 0.350, 0.500, 0.500, 0.500, 0.500, + 0.500, 0.500] + err0: [4.500, 4.500, 4.500, 2.500, 0.550, 0.300, 0.300, 0.400, 0.400, 0.400, + 0.450, 0.450, 0.550, 0.800, 4.000, 4.000, 4.000, 3.500, 3.000, 3.000, + 3.000, 3.000] + err1: [20.000, 25.000, 12.000, 7.000, 3.500, 3.000, 0.800, 0.400, 0.400, + 0.400, 0.450, 0.450, 0.550, 0.800, 4.000, 19.000, 30.000, 25.000, 16.500, + 12.000, 9.000, 6.500] + # Calculate Innovation@DerivedMetaData + - filter: Variable Assignment + assignments: + - name: Innovation@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsFunction/Arithmetic + channels: *{{observation_from_jcb}}_simulated_channels + options: + variables: + - name: brightnessTemperature@ObsValue + channels: *{{observation_from_jcb}}_simulated_channels + - name: brightnessTemperature@HofX + channels: *{{observation_from_jcb}}_simulated_channels + coefs: [1, -1] + # Step 0-C: Assign Initial All-Sky Observation Error + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: assign error + error function: + name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + # Step 1: Remove Observations from the Edge of the Scan + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: 1-22 + where: + - variable: + name: MetaData/sensorScanPosition + is_in: 7-90 + actions: + - name: set + flag: ScanEdgeRemoval + - name: reject + + # Step 2: Data Thinning + - filter: Gaussian Thinning + horizontal_mesh: 145 + use_reduced_horizontal_grid: true + distance_norm: geodesic + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + actions: + - name: set + flag: Thinning + - name: reject + + # Step 3A: CLW Retrieval Check (observation_based) + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: 1-7, 16-22 + test variables: + - name: CLWRetFromObs@DerivedMetaData + maxvalue: 999.0 + actions: + - name: set + flag: CLWRetrievalCheck + - name: reject + + # Step 3B: CLW Retrieval Check (background_based) + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: 1-7, 16-22 + test variables: + - name: CLWRetFromBkg@DerivedMetaData + maxvalue: 999.0 + actions: + - name: set + flag: CLWRetrievalCheck + - name: reject + + # Step 4: Window Channel Sanity Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: 1-7, 16, 17-22 + test variables: + - name: Innovation@DerivedMetaData + channels: 1, 2, 5-7, 16 + maxvalue: 200.0 + minvalue: -200.0 + flag all filter variables if any test variable is out of bounds: true + actions: + - name: set + flag: WindowChannelExtremeResidual + - name: reject + + # Step 5: Hydrometeor Check (cloud/precipitation affected chanels) + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/HydrometeorCheckATMS + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: HydrometeorCheckATMS@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_clearsky: [4.500, 4.500, 4.500, 2.500, 0.550, 0.300, 0.300, 0.400, + 0.400, 0.400, 0.450, 0.450, 0.550, 0.800, 4.000, 4.000, 4.000, 3.500, + 3.000, 3.000, 3.000, 3.000] + clwret_function: + name: CLWRetFromObs@DerivedMetaData + obserr_function: + name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: DerivedMetaData/HydrometeorCheckATMS + channels: *{{observation_from_jcb}}_simulated_channels + maxvalue: 0.0 + actions: + - name: set + flag: HydrometeorCheck + ignore: rejected observations + - name: reject + + # Step 6: Observation Error Inflation based on Topography Check + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorTopo@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorFactorTopoRad@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsErrorFactorTopo@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + # Step 7: Obs Error Inflation based on TOA Transmittancec Check + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorTransmitTop@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorFactorTransmitTopRad@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsErrorFactorTransmitTop@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + # Step 8: Observation Error Inflation based on Surface Jacobian Check + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorSurfJacobian@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorFactorSurfJacobianRad@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + obserr_demisf: [0.010, 0.020, 0.015, 0.020, 0.200] + obserr_dtempf: [0.500, 2.000, 1.000, 2.000, 4.500] + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsErrorFactorSurfJacobian@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + # Step 9: Situation Dependent Check + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorSituDepend@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorFactorSituDependMW@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + clwbkg_function: + name: CLWRetFromBkg@DerivedMetaData + clwobs_function: + name: CLWRetFromObs@DerivedMetaData + scatobs_function: + name: SIRetFromObs@DerivedMetaData + clwmatchidx_function: + name: CLWMatchIndex@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + obserr_function: + name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + obserr_clearsky: [4.500, 4.500, 4.500, 2.500, 0.550, 0.300, 0.300, 0.400, + 0.400, 0.400, 0.450, 0.450, 0.550, 0.800, 4.000, 4.000, 4.000, 3.500, + 3.000, 3.000, 3.000, 3.000] + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsErrorFactorSituDepend@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + # Step 10: Gross check + # Remove data if abs(Obs-HofX) > absolute threhold + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorLat@DerivedMetaData + type: float + function: + name: ObsErrorFactorLatRad@ObsFunction + options: + latitude_parameters: [25.0, 0.25, 0.04, 3.0] + - filter: Variable Assignment + assignments: + - name: ObsErrorBound@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorBoundMW@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_latitude: + name: ObsErrorFactorLat@DerivedMetaData + obserr_bound_transmittop: + name: ObsErrorFactorTransmitTop@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_topo: + name: ObsErrorFactorTopo@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + obserr_function: + name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + threhold: 3 + obserr_bound_max: [4.5, 4.5, 3.0, 3.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, + 1.0, 1.0, 2.0, 4.5, 4.5, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0] + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + function absolute threshold: + - name: ObsErrorBound@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + actions: + - name: set + flag: GrossCheck + ignore: rejected observations + - name: reject + + # Step 11: Inter-Channel Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: InterChannelConsistencyCheck@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use passive_bc: true + sensor: *{{observation_from_jcb}}_sensor_id + use_flag: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 4, 1, 1, 1, 1, 1, 1, + 1] + maxvalue: 1.0e-12 + actions: + - name: set + flag: InterChannelConsistency + ignore: rejected observations + - name: reject + + # Step 12: Useflag Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/ChannelUseflagCheckRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 4, 1, 1, 1, 1, 1, 1, + 1] + minvalue: 1.0e-12 + actions: + - name: set + flag: UseflagCheck + ignore: rejected observations + - name: reject + + # Observation Localizations (LocalEnsembleDA) + # ------------------------------------------- + obs localizations: + - localization method: Horizontal Gaspari-Cohn + lengthscale: 1250e3 + max nobs: 10000 + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/atms_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/atms_npp.yaml.j2 new file mode 100644 index 000000000..6356796c5 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/atms_npp.yaml.j2 @@ -0,0 +1,564 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: ATMS NPP + distribution: + name: Halo + halo size: 1250e3 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [brightnessTemperature] + channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} + + # Observation Operator + # -------------------- + obs operator: + name: CRTM + Absorbers: [H2O, O3, CO2] + Clouds: [Water, Ice] + Cloud_Fraction: 1.0 + Cloud_Seeding: true + obs options: + Sensor_ID: &{{observation_from_jcb}}_sensor_id atms_npp + EndianType: little_endian + CoefficientPath: "{{crtm_coefficient_path}}" + linear obs operator: + Absorbers: [H2O, O3] + + # Observation Bias Correction (VarBC) + # ----------------------------------- + obs bias: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" + variational bc: + predictors: + - name: constant + - name: lapseRate + order: 2 + tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" + - name: lapseRate + tlapse: *{{observation_from_jcb}}_tlapse + - name: emissivityJacobian + - name: sensorScanAngle + order: 4 + - name: sensorScanAngle + order: 3 + - name: sensorScanAngle + order: 2 + - name: sensorScanAngle + covariance: + minimal required obs number: 20 + variance range: [1.0e-6, 10.0] + step size: 1.0e-4 + largest analysis variance: 10000.0 + prior: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiascovin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovin_suffix}}" + inflation: + ratio: 1.1 + ratio for small dataset: 2.0 + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" + + # Observation Pre Filters (QC) + # ---------------------------- + obs pre filters: + # Step 0-A: Create Diagnostic Flags + - filter: Create Diagnostic Flags + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + flags: + - name: ScanEdgeRemoval + initial value: false + force reinitialization: false + - name: Thinning + initial value: false + force reinitialization: false + - name: CLWRetrievalCheck + initial value: false + force reinitialization: false + - name: WindowChannelExtremeResidual + initial value: false + force reinitialization: false + - name: HydrometeorCheck + initial value: false + force reinitialization: false + - name: GrossCheck + initial value: false + force reinitialization: false + - name: InterChannelConsistency + initial value: false + force reinitialization: false + - name: UseflagCheck + initial value: false + force reinitialization: false + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Step 0-B: Calculate derived variables + # Calculate CLW retrieved from observation + - filter: Variable Assignment + assignments: + - name: CLWRetFromObs@DerivedMetaData + type: float + function: + name: CLWRetMW@ObsFunction + options: + clwret_ch238: 1 + clwret_ch314: 2 + clwret_types: [ObsValue] + + # Calculate CLW retrieved from observation + - filter: Variable Assignment + assignments: + - name: CLWRetFromBkg@DerivedMetaData + type: float + function: + name: CLWRetMW@ObsFunction + options: + clwret_ch238: 1 + clwret_ch314: 2 + clwret_types: [HofX] + + # Calculate symmetric retrieved CLW + - filter: Variable Assignment + assignments: + - name: CLWRetSymmetric@DerivedMetaData + type: float + value: 1000.0 + + - filter: Variable Assignment + where: + - variable: + name: CLWRetFromObs@DerivedMetaData + minvalue: 0. + maxvalue: 999. + - variable: + name: CLWRetFromBkg@DerivedMetaData + minvalue: 0. + maxvalue: 999. + where operator: and + assignments: + - name: CLWRetSymmetric@DerivedMetaData + type: float + function: + name: Arithmetic@ObsFunction + options: + variables: + - name: CLWRetFromObs@DerivedMetaData + - name: CLWRetFromBkg@DerivedMetaData + total coefficient: 0.5 + + # Calculate scattering index from observation + - filter: Variable Assignment + assignments: + - name: SIRetFromObs@DerivedMetaData + type: float + function: + name: SCATRetMW@ObsFunction + options: + scatret_ch238: 1 + scatret_ch314: 2 + scatret_ch890: 16 + scatret_types: [ObsValue] + + # Calculate CLW obs/bkg match index + - filter: Variable Assignment + assignments: + - name: CLWMatchIndex@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: CLWMatchIndexMW@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + clwobs_function: + name: CLWRetFromObs@DerivedMetaData + clwbkg_function: + name: CLWRetFromBkg@DerivedMetaData + clwret_clearsky: [0.030, 0.030, 0.030, 0.020, 0.030, 0.080, 0.150, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.020, 0.030, 0.030, + 0.030, 0.030, 0.050, 0.100] + + # Calculate symmetric observation error + - filter: Variable Assignment + assignments: + - name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorModelRamp@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + xvar: + name: CLWRetSymmetric@DerivedMetaData + x0: [0.030, 0.030, 0.030, 0.020, 0.030, 0.080, 0.150, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.000, 0.020, 0.030, 0.030, 0.030, 0.030, + 0.050, 0.100] + x1: [0.350, 0.380, 0.400, 0.450, 0.500, 1.000, 1.000, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.000, 0.350, 0.500, 0.500, 0.500, 0.500, + 0.500, 0.500] + err0: [4.500, 4.500, 4.500, 2.500, 0.550, 0.300, 0.300, 0.400, 0.400, 0.400, + 0.450, 0.450, 0.550, 0.800, 4.000, 4.000, 4.000, 3.500, 3.000, 3.000, + 3.000, 3.000] + err1: [20.000, 25.000, 12.000, 7.000, 3.500, 3.000, 0.800, 0.400, 0.400, + 0.400, 0.450, 0.450, 0.550, 0.800, 4.000, 19.000, 30.000, 25.000, 16.500, + 12.000, 9.000, 6.500] + + # Calculate Innovation@DerivedMetaData + - filter: Variable Assignment + assignments: + - name: Innovation@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsFunction/Arithmetic + channels: *{{observation_from_jcb}}_simulated_channels + options: + variables: + - name: brightnessTemperature@ObsValue + channels: *{{observation_from_jcb}}_simulated_channels + - name: brightnessTemperature@HofX + channels: *{{observation_from_jcb}}_simulated_channels + coefs: [1, -1] + + # Step 0-C: Assign Initial All-Sky Observation Error + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: assign error + error function: + name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 1: Remove Observations from the Edge of the Scan + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: 1-22 + where: + - variable: + name: MetaData/sensorScanPosition + is_in: 7-90 + actions: + - name: set + flag: ScanEdgeRemoval + - name: reject + + # Step 2: Data Thinning + - filter: Gaussian Thinning + horizontal_mesh: 145 + use_reduced_horizontal_grid: true + distance_norm: geodesic + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + actions: + - name: set + flag: Thinning + - name: reject + + # Step 3A: CLW Retrieval Check (observation_based) + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: 1-7, 16-22 + test variables: + - name: CLWRetFromObs@DerivedMetaData + maxvalue: 999.0 + actions: + - name: set + flag: CLWRetrievalCheck + - name: reject + + # Step 3B: CLW Retrieval Check (background_based) + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: 1-7, 16-22 + test variables: + - name: CLWRetFromBkg@DerivedMetaData + maxvalue: 999.0 + actions: + - name: set + flag: CLWRetrievalCheck + - name: reject + + # Step 4: Window Channel Sanity Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: 1-7, 16, 17-22 + test variables: + - name: Innovation@DerivedMetaData + channels: 1, 2, 5-7, 16 + maxvalue: 200.0 + minvalue: -200.0 + flag all filter variables if any test variable is out of bounds: true + actions: + - name: set + flag: WindowChannelExtremeResidual + - name: reject + + # Step 5: Hydrometeor Check (cloud/precipitation affected chanels) + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/HydrometeorCheckATMS + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: HydrometeorCheckATMS@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_clearsky: [4.500, 4.500, 4.500, 2.500, 0.550, 0.300, 0.300, 0.400, + 0.400, 0.400, 0.450, 0.450, 0.550, 0.800, 4.000, 4.000, 4.000, 3.500, + 3.000, 3.000, 3.000, 3.000] + clwret_function: + name: CLWRetFromObs@DerivedMetaData + obserr_function: + name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: DerivedMetaData/HydrometeorCheckATMS + channels: *{{observation_from_jcb}}_simulated_channels + maxvalue: 0.0 + actions: + - name: set + flag: HydrometeorCheck + ignore: rejected observations + - name: reject + + # Step 6: Observation Error Inflation based on Topography Check + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorTopo@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorFactorTopoRad@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsErrorFactorTopo@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 7: Obs Error Inflation based on TOA Transmittancec Check + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorTransmitTop@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorFactorTransmitTopRad@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsErrorFactorTransmitTop@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 8: Observation Error Inflation based on Surface Jacobian Check + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorSurfJacobian@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorFactorSurfJacobianRad@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + obserr_demisf: [0.010, 0.020, 0.015, 0.020, 0.200] + obserr_dtempf: [0.500, 2.000, 1.000, 2.000, 4.500] + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsErrorFactorSurfJacobian@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 9: Situation Dependent Check + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorSituDepend@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorFactorSituDependMW@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + clwbkg_function: + name: CLWRetFromBkg@DerivedMetaData + clwobs_function: + name: CLWRetFromObs@DerivedMetaData + scatobs_function: + name: SIRetFromObs@DerivedMetaData + clwmatchidx_function: + name: CLWMatchIndex@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + obserr_function: + name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + obserr_clearsky: [4.500, 4.500, 4.500, 2.500, 0.550, 0.300, 0.300, 0.400, + 0.400, 0.400, 0.450, 0.450, 0.550, 0.800, 4.000, 4.000, 4.000, 3.500, + 3.000, 3.000, 3.000, 3.000] + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsErrorFactorSituDepend@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 10: Gross check + # Remove data if abs(Obs-HofX) > absolute threhold + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorLat@DerivedMetaData + type: float + function: + name: ObsErrorFactorLatRad@ObsFunction + options: + latitude_parameters: [25.0, 0.25, 0.04, 3.0] + + - filter: Variable Assignment + assignments: + - name: ObsErrorBound@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorBoundMW@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_latitude: + name: ObsErrorFactorLat@DerivedMetaData + obserr_bound_transmittop: + name: ObsErrorFactorTransmitTop@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_topo: + name: ObsErrorFactorTopo@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + obserr_function: + name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + threhold: 3 + obserr_bound_max: [4.5, 4.5, 3.0, 3.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, + 1.0, 1.0, 2.0, 4.5, 4.5, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0] + + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + function absolute threshold: + - name: ObsErrorBound@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + actions: + - name: set + flag: GrossCheck + ignore: rejected observations + - name: reject + + # Step 11: Inter-Channel Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: InterChannelConsistencyCheck@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use passive_bc: true + sensor: *{{observation_from_jcb}}_sensor_id + use_flag: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 4, 1, 1, 1, 1, 1, 1, + 1] + maxvalue: 1.0e-12 + actions: + - name: set + flag: InterChannelConsistency + ignore: rejected observations + - name: reject + + # Step 12: Useflag Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/ChannelUseflagCheckRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 4, 1, 1, 1, 1, 1, 1, + 1] + minvalue: 1.0e-12 + actions: + - name: set + flag: UseflagCheck + ignore: rejected observations + - name: reject + +# Observation Localizations (LocalEnsembleDA) + # ------------------------------------------- + obs localizations: + - localization method: Horizontal Gaspari-Cohn + lengthscale: 1250e3 + max nobs: 10000 + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml.j2 new file mode 100644 index 000000000..0ca33b1f2 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml.j2 @@ -0,0 +1,373 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: surface_ps + distribution: + name: Halo + halo size: 1250e3 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [stationPressure] + + # Observation Operator + # -------------------- + obs operator: + name: SfcPCorrected + variables: + - name: stationPressure + da_psfc_scheme: GSI + station_altitude: height + geovar_sfc_geomz: surface_altitude + geovar_geomz: geopotential_height + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: Identity + variables: + - name: stationPressure + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Initial Error Assignments for SFC Observations + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [181] + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + round_to_the_nearest_integer: true + xvar: + name: ObsValue/stationPressure + xvals: [80000, 75000, 70000, 65000, 60000, 55000] + errors: [110, 120, 120, 120, 120, 1.0e+11] + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [187] + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + round_to_the_nearest_integer: true + xvar: + name: ObsValue/stationPressure + xvals: [85000, 80000, 75000, 70000, 65000, 60000, 55000] + errors: [120, 140, 140, 140, 140, 140, 1.0e+11] + # Initial Error Assignments for SFCSHIP Observations + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [180] + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + round_to_the_nearest_integer: true + xvar: + name: ObsValue/stationPressure + xvals: [60000, 55000] + errors: [130, 1.0e+11] + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [183] + action: + name: assign error + error parameter: 1.0e+11 + + # Initial Error Assignments for Radiosonde + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [120] + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + round_to_the_nearest_integer: true + xvar: + name: ObsValue/stationPressure + xvals: [80000, 75000, 70000, 65000, 60000, 55000] + errors: [110, 120, 120, 120, 120, 1.0e+11] + # Create PreQC group variable (pqm in GSI read_prepbufr) + - filter: Variable Assignment + assignments: + - name: InputObsError/stationPressure + type: float + source variable: ObsErrorData/stationPressure + + # Set observation quality-realted variables + # Create PreQC group variable (pqm in GSI read_prepbufr) + - filter: Variable Assignment + assignments: + - name: PreQC/stationPressure + type: int + source variable: QualityMarker/stationPressure + + # Create PreUseFlag group variable (usage in GSI read_prepbufr) + # Initialize + - filter: Variable Assignment + assignments: + - name: PreUseFlag/stationPressure + type: int + source variable: PreQC/stationPressure + + - filter: Variable Assignment + where: + - variable: + name: PreUseFlag/stationPressure + is_in: 1-15 + assignments: + - name: PreUseFlag/stationPressure + value: 0 + # Re-assignment + - filter: Variable Assignment + where: + - variable: + name: ObsType/stationPressure + is_in: 183 + assignments: + - name: PreUseFlag/stationPressure + value: 100 + + - filter: Variable Assignment + where: + - variable: + name: ObsValue/stationPressure + is_defined: + - variable: + name: ObsValue/stationPressure + maxvalue: 50000.00 + where operator: and + assignments: + - name: PreUseFlag/stationPressure + value: 100 + + - filter: Variable Assignment + where: + - variable: + name: PreQC/stationPressure + is_in: 9, 12, 15 + assignments: + - name: PreUseFlag/stationPressure + value: 100 + + - filter: Variable Assignment + where: + - variable: + name: PreQC/stationPressure + is_in: 4-15 + assignments: + - name: PreUseFlag/stationPressure + value: 101 + + ## Observation range sanity check + #- filter: Bounds Check + # filter variables: + # - name: stationPressure + # minvalue: 37499.0 + # maxvalue: 106999.0 + # action: + # name: reject + ## Reject all ObsType 183 + #- filter: RejectList + # where: + # - variable: + # name: ObsType/stationPressure + # is_in: 183 + ## Reject surface pressure below 500 hPa + #- filter: Bounds Check + # filter variables: + # - name: stationPressure + # minvalue: 50000.00 + # action: + # name: reject + #- filter: RejectList + # where: + # - variable: + # name: PreQC/stationPressure + # is_in: 4-15 + # Inflate obs error based on obs type + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: PreQC/stationPressure + is_in: 3, 7 + action: + name: inflate error + inflation factor: 1.2 + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Calculate obs error inflation factors for duplicated observations at the same location + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorDuplicateCheck/stationPressure + type: float + function: + name: ObsFunction/ObsErrorFactorDuplicateCheck + options: + use_air_pressure: false + variable: stationPressure + + # Reduce effective observation error based on obs type and subtype + # In this case: reduce effective obs error for buoy + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: + name: ObsType/stationPressure + is_in: 180 + - variable: + name: ObsSubType/stationPressure + is_in: 0 + action: + name: inflate error + inflation factor: 0.7 + + # Calculate obs error inflation factors for large discrepancies between model and observations + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorSfcPressure/stationPressure + type: float + function: + name: ObsFunction/ObsErrorFactorSfcPressure + options: + geovar_sfc_geomz: surface_altitude + geovar_geomz: geopotential_height + station_altitude: height + + # Inflate surface pressure observation based on discrepancies between + # model and observations due to terrian + - filter: Perform Action + filter variables: + - name: stationPressure + action: + name: inflate error + inflation variable: + name: ObsErrorFactorSfcPressure/stationPressure + + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/Innovation + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsValue/stationPressure + - name: HofX/stationPressure + coefs: [1, -1] + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ObsErrorBoundSfcPressure1 + type: float + function: + name: ObsFunction/ObsErrorBoundConventional + options: + obsvar: stationPressure + obserr_bound_min: 100 + obserr_bound_max: 300 + obserr_bound_factor: 5.0 + + - filter: Background Check + filter variables: + - name: stationPressure + where: + - variable: PreQC/stationPressure + is_not_in: 3 + function absolute threshold: + - name: DerivedMetaData/ObsErrorBoundSfcPressure1 + action: + name: reject + + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ObsErrorBoundSfcPressure2 + type: float + function: + name: ObsFunction/ObsErrorBoundConventional + options: + obsvar: stationPressure + obserr_bound_min: 100 + obserr_bound_max: 300 + obserr_bound_factor: 3.5 + + - filter: Background Check + filter variables: + - name: stationPressure + where: + - variable: PreQC/stationPressure + is_in: 3 + function absolute threshold: + - name: DerivedMetaData/ObsErrorBoundSfcPressure2 + action: + name: reject + + # Inflate obs error based on duplicate check + - filter: Perform Action + filter variables: + - name: stationPressure + action: + name: inflate error + inflation variable: + name: ObsErrorFactorDuplicateCheck/stationPressure + + # Reject data based on PreUseFlag (usage in GSI) + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: PreUseFlag/stationPressure + is_not_in: 0, 1 + action: + name: reject + # End of Filters + + # Observation Localizations (LocalEnsembleDA) + # ------------------------------------------- + obs localizations: + - localization method: Horizontal Gaspari-Cohn + lengthscale: 1250e3 + max nobs: 10000 + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/mtiasi_metop-a.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/mtiasi_metop-a.yaml.j2 new file mode 100644 index 000000000..efc4535b8 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/mtiasi_metop-a.yaml.j2 @@ -0,0 +1,588 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: IASI METOP-A + distribution: + name: Halo + halo size: 1250e3 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + observed variables: [radiance] + simulated variables: [brightnessTemperature] + derived variables: [brightnessTemperature] + channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} + + # Observation Operator + # -------------------- + obs operator: + name: CRTM + Absorbers: [H2O, O3, CO2] + obs options: + Sensor_ID: &{{observation_from_jcb}}_sensor_id iasi_metop-a + EndianType: little_endian + CoefficientPath: "{{crtm_coefficient_path}}" + linear obs operator: + Absorbers: [H2O, O3] + + # Observation Bias Correction (VarBC) + # ----------------------------------- + obs bias: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" + variational bc: + predictors: + - name: constant + - name: lapseRate + order: 2 + tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" + - name: lapseRate + tlapse: *{{observation_from_jcb}}_tlapse + - name: emissivityJacobian + - name: sensorScanAngle + order: 4 + - name: sensorScanAngle + order: 3 + - name: sensorScanAngle + order: 2 + - name: sensorScanAngle + covariance: + minimal required obs number: 20 + variance range: [1.0e-6, 10.0] + step size: 1.0e-4 + largest analysis variance: 10000.0 + prior: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiascovin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovin_suffix}}" + inflation: + ratio: 1.1 + ratio for small dataset: 2.0 + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" + + # Observation Pre Filters (QC) + # ---------------------------- + obs pre filters: + # Step 0-A: Create Diagnostic Flags + #- filter: Create Diagnostic Flags + # filter variables: + # - name: brightnessTemperature + # channels: *{{observation_from_jcb}}_simulated_channels + # flags: + # - name: ScanEdgeRemoval + # initial value: false + # force reinitialization: false + # - name: Thinning + # initial value: false + # force reinitialization: false + # - name: ShortwaveIRCheck + # initial value: false + # force reinitialization: false + # - name: ObsValueRangeCheck + # initial value: false + # force reinitialization: false + # - name: CloudDetection + # initial value: false + # force reinitialization: false + # - name: NSSTCheck + # initial value: false + # force reinitialization: false + # - name: GrossCheck + # initial value: false + # force reinitialization: false + # - name: InterChannelConsistency + # initial value: false + # force reinitialization: false + # - name: UseFlagCheck + # initial value: false + # force reinitialization: false + # Step 0-B: Create Derived Variables + # Assign channel wavenumbers in m-1 + - filter: Variable Assignment + assignments: + - name: MetaData/sensorCentralWavenumber + type: float + channels: *{{observation_from_jcb}}_simulated_channels + function: + name: ObsFunction/LinearCombination + options: + variables: + - name: ObsValue/radiance + channels: *{{observation_from_jcb}}_simulated_channels + coefs: [25.0] + intercept: 64475.0 + use channel numbers: true + + # Transform radiance to brightness temperature + - filter: Variable Transforms + Transform: SatBrightnessTempFromRad + transform from: + name: ObsValue/radiance + channels: *{{observation_from_jcb}}_simulated_channels + spectral variable: + name: MetaData/sensorCentralWavenumber + channels: *{{observation_from_jcb}}_simulated_channels + radiance units: wavenumber + planck1: 1.191042953e-16 + planck2: 1.4387774e-2 + + # Step 0-C: Assign Observation Error + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: assign error + error parameter vector: &{{observation_from_jcb}}_oberr [1.38, 0.81, 0.75, 0.79, 0.72, 0.74, + 0.68, 0.72, 0.65, 0.65, 0.65, 0.69, 0.64, 0.64, 0.65, 0.67, 0.62, 0.61, 0.62, + 0.64, 0.59, 0.76, 1.22, 0.78, 0.64, 0.62, 0.61, 0.69, 0.65, 0.59, 0.61, 0.59, + 0.68, 0.62, 0.68, 4.38, 3.05, 2.31, 1.56, 1.33, 1.58, 0.93, 1.67, 0.72, 0.57, + 0.58, 0.55, 0.68, 0.59, 0.68, 0.59, 0.65, 0.58, 0.62, 0.64, 0.58, 0.64, 0.55, + 0.64, 0.5, 0.82, 0.59, 0.62, 0.51, 0.64, 0.52, 0.51, 0.51, 0.76, 0.52, 0.57, + 0.55, 0.69, 0.58, 0.65, 0.61, 0.59, 0.64, 0.76, 0.72, 1.05, 0.75, 0.51, 0.65, + 1.3, 0.69, 0.93, 1.49, 1.12, 0.68, 0.66, 0.67, 0.59, 0.59, 0.69, 0.67, 0.64, + 0.62, 0.72, 0.69, 0.66, 0.79, 0.78, 0.74, 0.88, 0.77, 0.88, 0.86, 1, 0.87, + 0.85, 0.88, 0.84, 0.84, 0.84, 0.8, 0.8, 0.87, 0.98, 0.52, 0.65, 0.69, 0.61, + 0.6, 0.67, 0.79, 0.62, 0.66, 0.7, 0.65, 0.62, 0.61, 0.62, 0.53, 0.6, 0.68, + 0.95, 0.63, 0.97, 0.65, 0.98, 0.58, 0.73, 0.65, 0.85, 0.99, 0.76, 0.85, 0.97, + 0.77, 0.62, 0.63, 1.21, 1.41, 1.55, 1.78, 1.35, 1.14, 1.69, 1.79, 1.46, 1.63, + 1.94, 2.01, 1.24, 1.76, 1.26, 1.47, 1.9, 1.66, 2.13, 1.49, 1.52, 1.55, 1.96, + 2.31, 2.33, 2.32, 2.31, 2.33, 2.23, 2.33, 1.84, 2.29, 2.28, 2.28, 2.28, 2.26, + 2.26, 2.26, 2.27, 2.24, 2.23, 2.24, 2.26, 2.28, 2.28, 2.3, 2.15, 2.31, 2.37, + 2.27, 2.29, 2.29, 2.23, 2.28, 2.32, 2.32, 2.31, 2.32, 2.32, 2.31, 2.31, 2.28, + 2.29, 2.28, 2.26, 2.29, 2.27, 2.26, 2.25, 2.27, 2.24, 2.21, 2.24, 2.17, 2.18, + 2.17, 2.21, 1.99, 2.16, 2.2, 2.13, 2.12, 2.13, 2.1, 2.12, 2.11, 2.09, 2.09, + 2.08, 2.09, 2.04, 2.04, 2.1, 2.01, 2.05, 2.03, 2.06, 1.98, 1.95, 1.94, 1.91, + 1.7, 1.76, 1.77, 1.83, 2.04, 1.91, 1.99, 1.99, 2.07, 2.02, 2.04, 2.1, 2.06, + 2.18, 2.21, 2.24, 2.23, 2.23, 1.98, 2.2, 2.18, 2.18, 2.21, 2.23, 2.24, 2.24, + 2.25, 1.8, 2.24, 1.73, 1.73, 2.27, 1.67, 2.21, 1.72, 2.23, 2.23, 2.23, 2.24, + 2.23, 2.12, 2.17, 1.74, 2.02, 1.88, 1.67, 1.73, 1.83, 1.82, 1.73, 1.83, 2.19, + 1.84, 1.89, 1.6, 1.71, 1.86, 1.85, 1.84, 1.87, 1.91, 1.52, 1.95, 1.87, 1.89, + 1.91, 1.91, 1.93, 1.9, 1.91, 1.9, 1.89, 1.89, 1.91, 1.9, 1.91, 1.91, 1.91, + 1.93, 1.94, 1.91, 1.92, 1.77, 1.91, 1.95, 1.19, 1.96, 1.98, 1.94, 1.55, 1.91, + 1.92, 1.92, 1.97, 1.93, 1.99, 1.86, 1.12, 1.93, 1.92, 1.95, 1.85, 1.84, 1.91, + 1.12, 1.82, 1.82, 1.95, 1.24, 1.94, 1.96, 1.21, 1.83, 1.96, 1.36, 1.96, 1.82, + 1.92, 1.68, 1.93, 1.23, 1.96, 1.93, 1.86, 1.41, 1.16, 1.6, 1.25, 1.2, 1.65, + 1.66, 1.87, 1.94, 1.96, 1.91, 1.25, 1.93, 1.91, 1.7, 0.99, 1.81, 1.92, 1.95, + 1.5, 1.47, 1.15, 1.58, 1.18, 1.82, 1.13, 1.83, 1.91, 1.26, 1.27, 1.91, 1.45, + 1.6, 1.29, 1.94, 1.94, 1.23, 1.95, 1.21, 1.94, 1.86, 1.9, 1.33, 1.75, 2.02, + 1.98, 2.03, 1.83, 1.5, 2.04, 2.02, 1.9, 2, 2.02, 1.95, 1.93, 1.95, 1.95, 1.99, + 2, 1.94, 1.96, 1.86, 1.92, 1.88, 1.86, 1.84, 1.87, 1.77, 1.89, 1.89, 1.88, + 1.94, 1.82, 1.79, 1.86, 2.06, 2.33, 1.88, 1.86, 1.81, 1.8, 1.8, 1.86, 1.9, + 2, 2.06, 2.1, 2.2, 2, 2.16, 1.98, 1.8, 1.8, 1.85, 1.75, 2.04, 2.19, 2.14, + 2.19, 1.86, 2.1, 2.11, 2.18, 2.03, 2.28, 2.19, 2.26, 2.26, 2.21, 2.21, 2.26, + 2.33, 2.27, 2.21, 2.12, 2.23, 2.26, 2.25, 1.88, 2.26, 2.24, 2.36, 2.29, 2.35, + 2.3, 2.27, 2.08, 2.05, 2.27, 2.28, 2.27, 2.28, 1.97, 2.25, 2.25, 2.25, 2.31, + 2.28, 2.27, 2.13, 2.24, 2.28, 2.28, 2.41, 2.34, 9.32, 2.28, 2.38, 2.27, 2.27, + 2.39, 2.11, 2.09, 2.1, 2.06, 2.12, 2.08, 2, 1.93, 2.02, 2.55, 1.54, 1.64, + 1.51, 1.55, 2.82, 2.92, 2.55, 2.37, 1.85, 1.6, 1.72, 1.74, 1.79, 1.9, 1.94, + 2, 2.04, 2.08, 2.12, 2.13, 2.16, 2.18, 2.18, 2.2, 2.2, 2.41, 2.39, 2.38, 2.4, + 2.42, 2.41, 2.43, 2.45, 2.43, 2.45, 2.43, 2.4, 2.44, 2.4, 2.42, 2.43, 2.45, + 2.45, 2.45, 2.46, 2.45, 2.45, 2.43, 2.51, 2.48, 2.48, 2.53, 2.46, 2.49, 2.5, + 2.5, 2.5, 2.52, 2.52, 2.54, 2.5, 2.48, 2.5, 2.55, 2.5, 2.48, 2.5, 2.5, 2.52, + 2.52, 2.48, 2.5, 2.5, 2.52, 2.46, 2.53, 9] + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Step 1: Remove Observations from the Edge of the Scan + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: MetaData/sensorScanPosition + is_in: 5-56 + action: + name: reject + # actions: + # - name: set + # flag: ScanEdgeRemoval + # - name: reject + # Step 2: Data Thinning + - filter: Gaussian Thinning + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + horizontal_mesh: 145 + use_reduced_horizontal_grid: true + distance_norm: geodesic + priority_variable: MetaData/fractionOfClearPixelsInFOV + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + action: + name: reject + # actions: + # - name: set + # flag: Thinning + # - name: reject + # Step 3: Wavenumber Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: 7024, 7027, 7029, 7032, 7038, 7043, 7046, 7049, 7069, 7072, 7076, + 7081, 7084, 7089, 7099, 7209, 7222, 7231, 7235, 7247, 7267, 7269, 7284, 7389, + 7419, 7423, 7424, 7426, 7428, 7431, 7436, 7444, 7475, 7549, 7584, 7665, 7666, + 7831, 7836, 7853, 7865, 7885, 7888, 7912, 7950, 7972, 7980, 7995, 8007, 8015, + 8055, 8078 + where: + - variable: + name: MetaData/solarZenithAngle + maxvalue: 88.9999 + - variable: + name: water_area_fraction@GeoVaLs + minvalue: 1.0e-12 + action: + name: reject + # actions: + # - name: set + # flag: ShortwaveIRCheck + # - name: reject + # Step 4: Observation Error Inflation based on Wavenumber + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsErrorFactorWavenumIR@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + # Step 5: Observation Range Sanity Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + minvalue: 50.00001 + maxvalue: 449.99999 + # maxvalue: 100.00000 + action: + name: reject + # actions: + # - name: set + # flag: ObsValueRangeCheck + # - name: reject + # Step 6: Topography Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorTopoRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + sensor: *{{observation_from_jcb}}_sensor_id + + # Step 7: Transmittance Top Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorTransmitTopRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + # Step 8: Cloud Detection Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/CloudDetectMinResidualIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: [1, -1, -1, -1, 1, -1, -1, -1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, + 1, 1, -1, 1, 1, 1, -1, -1, 1, 1, -1, 1, -1, 1, -1, 1, -1, -1, -1, -1, -1, + -1, -1, -1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, -1, + 1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, -1, -1, + 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, 1, 1, 1, + 1, 1, 1, -1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, 1, 1, -1, -1, -1, + 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, + 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, -1, 1, 1, 1, -1, 1, 1, -1, 1, -1, + 1, -1, -1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, -1, -1, -1, 1, 1, 1, -1, + -1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, 1, -1, 1, 1, -1, 1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, + -1, -1, 1, -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, 1, + -1, 1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, -1, -1, -1, -1, -1, 1, -1, 1, -1, + 1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, + -1, -1, 1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, + -1, -1, -1, -1, -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, 1, + -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1] + use_flag_clddet: [1, -1, -1, -1, 1, -1, -1, -1, 1, -1, 1, -1, 1, -1, 1, 1, + 1, -1, 1, 1, -1, 1, 1, 1, -1, -1, 1, 1, -1, 1, -1, 1, -1, 1, -1, -1, -1, + -1, -1, -1, -1, -1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, + 1, 1, -1, 1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, + -1, -1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, 1, + 1, 1, 1, 1, 1, -1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, 1, 1, -1, + -1, -1, 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, 1, + 1, 1, 1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, -1, 1, 1, 1, -1, 1, 1, + -1, 1, -1, 1, -1, -1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, -1, -1, -1, 1, + 1, 1, -1, -1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, 1, -1, 1, 1, -1, 1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 1, -1, 1, -1, 1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, + -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1] + obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] + error parameter vector: *{{observation_from_jcb}}_oberr + maxvalue: 1.0e-12 + action: + name: reject + # actions: + # - name: set + # flag: CloudDetection + # - name: reject + # Step 9: NSST Retrieval Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: NearSSTRetCheckIR@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: [1, -1, -1, -1, 1, -1, -1, -1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, + 1, 1, -1, 1, 1, 1, -1, -1, 1, 1, -1, 1, -1, 1, -1, 1, -1, -1, -1, -1, -1, + -1, -1, -1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, -1, + 1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, -1, -1, + 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, 1, 1, 1, + 1, 1, 1, -1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, 1, 1, -1, -1, -1, + 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, + 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, -1, 1, 1, 1, -1, 1, 1, -1, 1, -1, + 1, -1, -1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, -1, -1, -1, 1, 1, 1, -1, + -1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, 1, -1, 1, 1, -1, 1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, + -1, -1, 1, -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, 1, + -1, 1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, -1, -1, -1, -1, -1, 1, -1, 1, -1, + 1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, + -1, -1, 1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, + -1, -1, -1, -1, -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, 1, + -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1] + obserr_demisf: [0.01, 0.02, 0.03, 0.02, 0.03] + obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] + maxvalue: 1.0e-12 + action: + name: reject + # actions: + # - name: set + # flag: NSSTCheck + # - name: reject + # Step 10: Surface Jacobians Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorSurfJacobianRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_demisf: [0.01, 0.02, 0.03, 0.02, 0.03] + obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] + sensor: *{{observation_from_jcb}}_sensor_id + + # Step 11: Gross check + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + function absolute threshold: + - name: ObsFunction/ObsErrorBoundIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_latitude: + name: ObsFunction/ObsErrorFactorLatRad + options: + latitude_parameters: [25.0, 0.5, 0.04, 1.0] + obserr_bound_transmittop: + name: ObsFunction/ObsErrorFactorTransmitTopRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_max: [3.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 4.0, 2.0, 2.0, 2.0, + 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 4.0, 4.0, 3.5, + 2.5, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 3.5, 2.0, 2.0, 2.0, 2.0, 2.0, + 2.0, 3.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + 2.0, 2.0, 2.0, 2.5, 2.0, 2.5, 2.5, 3.0, 2.5, 2.5, 2.5, 2.5, 3.5, 2.5, 2.5, + 3.0, 3.5, 3.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.5, 4.5, 4.5, 4.5, 4.5, 4.0, + 4.5, 4.0, 4.0, 4.5, 2.5, 3.0, 2.5, 3.0, 2.5, 3.0, 2.0, 2.5, 2.5, 3.0, 3.0, + 2.5, 3.0, 3.0, 3.0, 2.5, 2.5, 4.0, 4.5, 4.5, 5.0, 4.0, 4.0, 5.0, 5.0, 5.0, + 5.0, 5.5, 5.5, 4.0, 5.0, 4.0, 4.5, 5.5, 5.5, 6.0, 4.5, 4.5, 4.0, 5.0, 5.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 5.5, 4.5, 6.0, 5.0, 5.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 5.0, 6.0, 6.0, 6.0, 4.0, 6.0, 6.0, 6.0, 6.0, 4.5, 6.0, 6.0, 4.5, + 6.0, 6.0, 6.0, 6.0, 6.0, 5.0, 6.0, 6.0, 6.0, 5.0, 6.0, 6.0, 5.0, 6.0, 5.0, + 6.0, 6.0, 6.0, 5.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0] + error parameter vector: *{{observation_from_jcb}}_oberr + action: + name: reject + # actions: + # - name: set + # flag: GrossCheck + # - name: reject + # Step 12: Useflag Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/ChannelUseflagCheckRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: [1, -1, -1, -1, 1, -1, -1, -1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, + 1, 1, -1, 1, 1, 1, -1, -1, 1, 1, -1, 1, -1, 1, -1, 1, -1, -1, -1, -1, -1, + -1, -1, -1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, -1, + 1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, -1, -1, + 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, 1, 1, 1, + 1, 1, 1, -1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, 1, 1, -1, -1, -1, + 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, + 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, -1, 1, 1, 1, -1, 1, 1, -1, 1, -1, + 1, -1, -1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, -1, -1, -1, 1, 1, 1, -1, + -1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, 1, -1, 1, 1, -1, 1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, + -1, -1, 1, -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, 1, + -1, 1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, -1, -1, -1, -1, -1, 1, -1, 1, -1, + 1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, + -1, -1, 1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, + -1, -1, -1, -1, -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, 1, + -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1] + minvalue: 1.0e-12 + action: + name: reject + # actions: + # - name: set + # flag: UseFlagCheck + # - name: reject + + # Observation Localizations (LocalEnsembleDA) + # ------------------------------------------- + obs localizations: + - localization method: Horizontal Gaspari-Cohn + lengthscale: 1250e3 + max nobs: 10000 + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/mtiasi_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/mtiasi_metop-b.yaml.j2 new file mode 100644 index 000000000..8f2f26103 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/mtiasi_metop-b.yaml.j2 @@ -0,0 +1,588 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: IASI METOP-B + distribution: + name: Halo + halo size: 1250e3 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + observed variables: [radiance] + simulated variables: [brightnessTemperature] + derived variables: [brightnessTemperature] + channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} + + # Observation Operator + # -------------------- + obs operator: + name: CRTM + Absorbers: [H2O, O3, CO2] + obs options: + Sensor_ID: &{{observation_from_jcb}}_sensor_id iasi_metop-b + EndianType: little_endian + CoefficientPath: "{{crtm_coefficient_path}}" + linear obs operator: + Absorbers: [H2O, O3] + + # Observation Bias Correction (VarBC) + # ----------------------------------- + obs bias: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" + variational bc: + predictors: + - name: constant + - name: lapseRate + order: 2 + tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" + - name: lapseRate + tlapse: *{{observation_from_jcb}}_tlapse + - name: emissivityJacobian + - name: sensorScanAngle + order: 4 + - name: sensorScanAngle + order: 3 + - name: sensorScanAngle + order: 2 + - name: sensorScanAngle + covariance: + minimal required obs number: 20 + variance range: [1.0e-6, 10.0] + step size: 1.0e-4 + largest analysis variance: 10000.0 + prior: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiascovin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovin_suffix}}" + inflation: + ratio: 1.1 + ratio for small dataset: 2.0 + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" + + # Observation Pre Filters (QC) + # ---------------------------- + obs pre filters: + # Step 0-A: Create Diagnostic Flags + #- filter: Create Diagnostic Flags + # filter variables: + # - name: brightnessTemperature + # channels: *{{observation_from_jcb}}_simulated_channels + # flags: + # - name: ScanEdgeRemoval + # initial value: false + # force reinitialization: false + # - name: Thinning + # initial value: false + # force reinitialization: false + # - name: ShortwaveIRCheck + # initial value: false + # force reinitialization: false + # - name: ObsValueRangeCheck + # initial value: false + # force reinitialization: false + # - name: CloudDetection + # initial value: false + # force reinitialization: false + # - name: NSSTCheck + # initial value: false + # force reinitialization: false + # - name: GrossCheck + # initial value: false + # force reinitialization: false + # - name: InterChannelConsistency + # initial value: false + # force reinitialization: false + # - name: UseFlagCheck + # initial value: false + # force reinitialization: false + # Step 0-B: Create Derived Variables + # Assign channel wavenumbers in m-1 + - filter: Variable Assignment + assignments: + - name: MetaData/sensorCentralWavenumber + type: float + channels: *{{observation_from_jcb}}_simulated_channels + function: + name: ObsFunction/LinearCombination + options: + variables: + - name: ObsValue/radiance + channels: *{{observation_from_jcb}}_simulated_channels + coefs: [25.0] + intercept: 64475.0 + use channel numbers: true + + # Transform radiance to brightness temperature + - filter: Variable Transforms + Transform: SatBrightnessTempFromRad + transform from: + name: ObsValue/radiance + channels: *{{observation_from_jcb}}_simulated_channels + spectral variable: + name: MetaData/sensorCentralWavenumber + channels: *{{observation_from_jcb}}_simulated_channels + radiance units: wavenumber + planck1: 1.191042953e-16 + planck2: 1.4387774e-2 + + # Step 0-C: Assign Observation Error + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: assign error + error parameter vector: &{{observation_from_jcb}}_oberr [1.38, 0.81, 0.75, 0.79, 0.72, 0.74, + 0.68, 0.72, 0.65, 0.65, 0.65, 0.69, 0.64, 0.64, 0.65, 0.67, 0.62, 0.61, 0.62, + 0.64, 0.59, 0.76, 1.22, 0.78, 0.64, 0.62, 0.61, 0.69, 0.65, 0.59, 0.61, 0.59, + 0.68, 0.62, 0.68, 4.38, 3.05, 2.31, 1.56, 1.33, 1.58, 0.93, 1.67, 0.72, 0.57, + 0.58, 0.55, 0.68, 0.59, 0.68, 0.59, 0.65, 0.58, 0.62, 0.64, 0.58, 0.64, 0.55, + 0.64, 0.5, 0.82, 0.59, 0.62, 0.51, 0.64, 0.52, 0.51, 0.51, 0.76, 0.52, 0.57, + 0.55, 0.69, 0.58, 0.65, 0.61, 0.59, 0.64, 0.76, 0.72, 1.05, 0.75, 0.51, 0.65, + 1.3, 0.69, 0.93, 1.49, 1.12, 0.68, 0.66, 0.67, 0.59, 0.59, 0.69, 0.67, 0.64, + 0.62, 0.72, 0.69, 0.66, 0.79, 0.78, 0.74, 0.88, 0.77, 0.88, 0.86, 1, 0.87, + 0.85, 0.88, 0.84, 0.84, 0.84, 0.8, 0.8, 0.87, 0.98, 0.52, 0.65, 0.69, 0.61, + 0.6, 0.67, 0.79, 0.62, 0.66, 0.7, 0.65, 0.62, 0.61, 0.62, 0.53, 0.6, 0.68, + 0.95, 0.63, 0.97, 0.65, 0.98, 0.58, 0.73, 0.65, 0.85, 0.99, 0.76, 0.85, 0.97, + 0.77, 0.62, 0.63, 1.21, 1.41, 1.55, 1.78, 1.35, 1.14, 1.69, 1.79, 1.46, 1.63, + 1.94, 2.01, 1.24, 1.76, 1.26, 1.47, 1.9, 1.66, 2.13, 1.49, 1.52, 1.55, 1.96, + 2.31, 2.33, 2.32, 2.31, 2.33, 2.23, 2.33, 1.84, 2.29, 2.28, 2.28, 2.28, 2.26, + 2.26, 2.26, 2.27, 2.24, 2.23, 2.24, 2.26, 2.28, 2.28, 2.3, 2.15, 2.31, 2.37, + 2.27, 2.29, 2.29, 2.23, 2.28, 2.32, 2.32, 2.31, 2.32, 2.32, 2.31, 2.31, 2.28, + 2.29, 2.28, 2.26, 2.29, 2.27, 2.26, 2.25, 2.27, 2.24, 2.21, 2.24, 2.17, 2.18, + 2.17, 2.21, 1.99, 2.16, 2.2, 2.13, 2.12, 2.13, 2.1, 2.12, 2.11, 2.09, 2.09, + 2.08, 2.09, 2.04, 2.04, 2.1, 2.01, 2.05, 2.03, 2.06, 1.98, 1.95, 1.94, 1.91, + 1.7, 1.76, 1.77, 1.83, 2.04, 1.91, 1.99, 1.99, 2.07, 2.02, 2.04, 2.1, 2.06, + 2.18, 2.21, 2.24, 2.23, 2.23, 1.98, 2.2, 2.18, 2.18, 2.21, 2.23, 2.24, 2.24, + 2.25, 1.8, 2.24, 1.73, 1.73, 2.27, 1.67, 2.21, 1.72, 2.23, 2.23, 2.23, 2.24, + 2.23, 2.12, 2.17, 1.74, 2.02, 1.88, 1.67, 1.73, 1.83, 1.82, 1.73, 1.83, 2.19, + 1.84, 1.89, 1.6, 1.71, 1.86, 1.85, 1.84, 1.87, 1.91, 1.52, 1.95, 1.87, 1.89, + 1.91, 1.91, 1.93, 1.9, 1.91, 1.9, 1.89, 1.89, 1.91, 1.9, 1.91, 1.91, 1.91, + 1.93, 1.94, 1.91, 1.92, 1.77, 1.91, 1.95, 1.19, 1.96, 1.98, 1.94, 1.55, 1.91, + 1.92, 1.92, 1.97, 1.93, 1.99, 1.86, 1.12, 1.93, 1.92, 1.95, 1.85, 1.84, 1.91, + 1.12, 1.82, 1.82, 1.95, 1.24, 1.94, 1.96, 1.21, 1.83, 1.96, 1.36, 1.96, 1.82, + 1.92, 1.68, 1.93, 1.23, 1.96, 1.93, 1.86, 1.41, 1.16, 1.6, 1.25, 1.2, 1.65, + 1.66, 1.87, 1.94, 1.96, 1.91, 1.25, 1.93, 1.91, 1.7, 0.99, 1.81, 1.92, 1.95, + 1.5, 1.47, 1.15, 1.58, 1.18, 1.82, 1.13, 1.83, 1.91, 1.26, 1.27, 1.91, 1.45, + 1.6, 1.29, 1.94, 1.94, 1.23, 1.95, 1.21, 1.94, 1.86, 1.9, 1.33, 1.75, 2.02, + 1.98, 2.03, 1.83, 1.5, 2.04, 2.02, 1.9, 2, 2.02, 1.95, 1.93, 1.95, 1.95, 1.99, + 2, 1.94, 1.96, 1.86, 1.92, 1.88, 1.86, 1.84, 1.87, 1.77, 1.89, 1.89, 1.88, + 1.94, 1.82, 1.79, 1.86, 2.06, 2.33, 1.88, 1.86, 1.81, 1.8, 1.8, 1.86, 1.9, + 2, 2.06, 2.1, 2.2, 2, 2.16, 1.98, 1.8, 1.8, 1.85, 1.75, 2.04, 2.19, 2.14, + 2.19, 1.86, 2.1, 2.11, 2.18, 2.03, 2.28, 2.19, 2.26, 2.26, 2.21, 2.21, 2.26, + 2.33, 2.27, 2.21, 2.12, 2.23, 2.26, 2.25, 1.88, 2.26, 2.24, 2.36, 2.29, 2.35, + 2.3, 2.27, 2.08, 2.05, 2.27, 2.28, 2.27, 2.28, 1.97, 2.25, 2.25, 2.25, 2.31, + 2.28, 2.27, 2.13, 2.24, 2.28, 2.28, 2.41, 2.34, 9.32, 2.28, 2.38, 2.27, 2.27, + 2.39, 2.11, 2.09, 2.1, 2.06, 2.12, 2.08, 2, 1.93, 2.02, 2.55, 1.54, 1.64, + 1.51, 1.55, 2.82, 2.92, 2.55, 2.37, 1.85, 1.6, 1.72, 1.74, 1.79, 1.9, 1.94, + 2, 2.04, 2.08, 2.12, 2.13, 2.16, 2.18, 2.18, 2.2, 2.2, 2.41, 2.39, 2.38, 2.4, + 2.42, 2.41, 2.43, 2.45, 2.43, 2.45, 2.43, 2.4, 2.44, 2.4, 2.42, 2.43, 2.45, + 2.45, 2.45, 2.46, 2.45, 2.45, 2.43, 2.51, 2.48, 2.48, 2.53, 2.46, 2.49, 2.5, + 2.5, 2.5, 2.52, 2.52, 2.54, 2.5, 2.48, 2.5, 2.55, 2.5, 2.48, 2.5, 2.5, 2.52, + 2.52, 2.48, 2.5, 2.5, 2.52, 2.46, 2.53, 9] + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Step 1: Remove Observations from the Edge of the Scan + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: MetaData/sensorScanPosition + is_in: 5-56 + action: + name: reject + # actions: + # - name: set + # flag: ScanEdgeRemoval + # - name: reject + # Step 2: Data Thinning + - filter: Gaussian Thinning + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + horizontal_mesh: 145 + use_reduced_horizontal_grid: true + distance_norm: geodesic + priority_variable: MetaData/fractionOfClearPixelsInFOV + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + action: + name: reject + # actions: + # - name: set + # flag: Thinning + # - name: reject + # Step 3: Wavenumber Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: 7024, 7027, 7029, 7032, 7038, 7043, 7046, 7049, 7069, 7072, 7076, + 7081, 7084, 7089, 7099, 7209, 7222, 7231, 7235, 7247, 7267, 7269, 7284, 7389, + 7419, 7423, 7424, 7426, 7428, 7431, 7436, 7444, 7475, 7549, 7584, 7665, 7666, + 7831, 7836, 7853, 7865, 7885, 7888, 7912, 7950, 7972, 7980, 7995, 8007, 8015, + 8055, 8078 + where: + - variable: + name: MetaData/solarZenithAngle + maxvalue: 88.9999 + - variable: + name: water_area_fraction@GeoVaLs + minvalue: 1.0e-12 + action: + name: reject + # actions: + # - name: set + # flag: ShortwaveIRCheck + # - name: reject + # Step 4: Observation Error Inflation based on Wavenumber + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsErrorFactorWavenumIR@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + # Step 5: Observation Range Sanity Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + minvalue: 50.00001 + maxvalue: 449.99999 + # maxvalue: 100.00000 + action: + name: reject + # actions: + # - name: set + # flag: ObsValueRangeCheck + # - name: reject + # Step 6: Topography Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorTopoRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + sensor: *{{observation_from_jcb}}_sensor_id + + # Step 7: Transmittance Top Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorTransmitTopRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + # Step 8: Cloud Detection Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/CloudDetectMinResidualIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: [1, -1, -1, -1, 1, -1, -1, -1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, + 1, 1, -1, 1, 1, 1, -1, -1, 1, 1, -1, 1, -1, 1, -1, 1, -1, -1, -1, -1, -1, + -1, -1, -1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, -1, + 1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, -1, -1, + 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, 1, 1, 1, + 1, 1, 1, -1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, 1, 1, -1, -1, -1, + 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, + 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, -1, 1, 1, 1, -1, 1, 1, -1, 1, -1, + 1, -1, -1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, -1, -1, -1, 1, 1, 1, -1, + -1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, 1, -1, 1, 1, -1, 1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, + -1, -1, 1, -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, 1, + -1, 1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, -1, -1, -1, -1, -1, 1, -1, 1, -1, + 1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, + -1, -1, 1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, + -1, -1, -1, -1, -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, 1, + -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1] + use_flag_clddet: [1, -1, -1, -1, 1, -1, -1, -1, 1, -1, 1, -1, 1, -1, 1, 1, + 1, -1, 1, 1, -1, 1, 1, 1, -1, -1, 1, 1, -1, 1, -1, 1, -1, 1, -1, -1, -1, + -1, -1, -1, -1, -1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, + 1, 1, -1, 1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, + -1, -1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, 1, + 1, 1, 1, 1, 1, -1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, 1, 1, -1, + -1, -1, 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, 1, + 1, 1, 1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, -1, 1, 1, 1, -1, 1, 1, + -1, 1, -1, 1, -1, -1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, -1, -1, -1, 1, + 1, 1, -1, -1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, 1, -1, 1, 1, -1, 1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 1, -1, 1, -1, 1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, + -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1] + obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] + error parameter vector: *{{observation_from_jcb}}_oberr + maxvalue: 1.0e-12 + action: + name: reject + # actions: + # - name: set + # flag: CloudDetection + # - name: reject + # Step 9: NSST Retrieval Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: NearSSTRetCheckIR@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: [1, -1, -1, -1, 1, -1, -1, -1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, + 1, 1, -1, 1, 1, 1, -1, -1, 1, 1, -1, 1, -1, 1, -1, 1, -1, -1, -1, -1, -1, + -1, -1, -1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, -1, + 1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, -1, -1, + 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, 1, 1, 1, + 1, 1, 1, -1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, 1, 1, -1, -1, -1, + 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, + 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, -1, 1, 1, 1, -1, 1, 1, -1, 1, -1, + 1, -1, -1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, -1, -1, -1, 1, 1, 1, -1, + -1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, 1, -1, 1, 1, -1, 1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, + -1, -1, 1, -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, 1, + -1, 1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, -1, -1, -1, -1, -1, 1, -1, 1, -1, + 1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, + -1, -1, 1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, + -1, -1, -1, -1, -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, 1, + -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1] + obserr_demisf: [0.01, 0.02, 0.03, 0.02, 0.03] + obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] + maxvalue: 1.0e-12 + action: + name: reject + # actions: + # - name: set + # flag: NSSTCheck + # - name: reject + # Step 10: Surface Jacobians Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorSurfJacobianRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_demisf: [0.01, 0.02, 0.03, 0.02, 0.03] + obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] + sensor: *{{observation_from_jcb}}_sensor_id + + # Step 11: Gross check + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + function absolute threshold: + - name: ObsFunction/ObsErrorBoundIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_latitude: + name: ObsFunction/ObsErrorFactorLatRad + options: + latitude_parameters: [25.0, 0.5, 0.04, 1.0] + obserr_bound_transmittop: + name: ObsFunction/ObsErrorFactorTransmitTopRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_max: [3.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 4.0, 2.0, 2.0, 2.0, + 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 4.0, 4.0, 3.5, + 2.5, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 3.5, 2.0, 2.0, 2.0, 2.0, 2.0, + 2.0, 3.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, + 2.0, 2.0, 2.0, 2.5, 2.0, 2.5, 2.5, 3.0, 2.5, 2.5, 2.5, 2.5, 3.5, 2.5, 2.5, + 3.0, 3.5, 3.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.5, 4.5, 4.5, 4.5, 4.5, 4.0, + 4.5, 4.0, 4.0, 4.5, 2.5, 3.0, 2.5, 3.0, 2.5, 3.0, 2.0, 2.5, 2.5, 3.0, 3.0, + 2.5, 3.0, 3.0, 3.0, 2.5, 2.5, 4.0, 4.5, 4.5, 5.0, 4.0, 4.0, 5.0, 5.0, 5.0, + 5.0, 5.5, 5.5, 4.0, 5.0, 4.0, 4.5, 5.5, 5.5, 6.0, 4.5, 4.5, 4.0, 5.0, 5.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 5.5, 4.5, 6.0, 5.0, 5.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 5.0, 6.0, 6.0, 6.0, 4.0, 6.0, 6.0, 6.0, 6.0, 4.5, 6.0, 6.0, 4.5, + 6.0, 6.0, 6.0, 6.0, 6.0, 5.0, 6.0, 6.0, 6.0, 5.0, 6.0, 6.0, 5.0, 6.0, 5.0, + 6.0, 6.0, 6.0, 5.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, + 6.0, 6.0, 6.0, 6.0, 6.0] + error parameter vector: *{{observation_from_jcb}}_oberr + action: + name: reject + # actions: + # - name: set + # flag: GrossCheck + # - name: reject + # Step 12: Useflag Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/ChannelUseflagCheckRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: [1, -1, -1, -1, 1, -1, -1, -1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, + 1, 1, -1, 1, 1, 1, -1, -1, 1, 1, -1, 1, -1, 1, -1, 1, -1, -1, -1, -1, -1, + -1, -1, -1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, -1, + 1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, -1, -1, + 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, 1, 1, 1, + 1, 1, 1, -1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, 1, 1, -1, -1, -1, + 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, + 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, -1, 1, 1, 1, -1, 1, 1, -1, 1, -1, + 1, -1, -1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, -1, -1, -1, 1, 1, 1, -1, + -1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, 1, -1, 1, 1, -1, 1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, + -1, -1, 1, -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, 1, + -1, 1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, -1, -1, -1, -1, -1, 1, -1, 1, -1, + 1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, + -1, -1, 1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, + -1, -1, -1, -1, -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, 1, + -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1] + minvalue: 1.0e-12 + action: + name: reject + # actions: + # - name: set + # flag: UseFlagCheck + # - name: reject + + # Observation Localizations (LocalEnsembleDA) + # ------------------------------------------- + obs localizations: + - localization method: Horizontal Gaspari-Cohn + lengthscale: 1250e3 + max nobs: 10000 + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/ompsnp_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/ompsnp_npp.yaml.j2 new file mode 100644 index 000000000..488afd73b --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/ompsnp_npp.yaml.j2 @@ -0,0 +1,339 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: ompsnp_npp + distribution: + name: Halo + halo size: 1250e3 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsgrouping: + group variables: ["latitude"] + sort variable: "pressure" + sort order: "ascending" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [ozoneLayer] + + #obs operator: + # name: AtmVertInterpLay + # geovals: [mole_fraction_of_ozone_in_air] + # coefficients: [0.007886131] # convert from ppmv to DU + # nlevels: [22] + + # Observation Operator + # -------------------- + obs operator: + name: ColumnRetrieval + nlayers_retrieval: 1 + tracer variables: [mole_fraction_of_ozone_in_air] + isApriori: false + isAveragingKernel: false + totalNoVertice: false + stretchVertices: none #options: top, bottom, topbottom, none + # model units coeff: 2.240013904035E-3 # this number to match the gsihofx values + model units coeff: 2.241398632746E-3 # this number to match the gsihofx values (use this) + # the actual scientific conversion factor is + # 2.1415E-3 kg[O3]/m-2 to DU + # so the name of the geovals + # is also likely wrong, as it apprears to be a mass + # fraction given the conversion factor needed + + # Observation Pre Filters (QC) + # ---------------------------- + obs pre filters: + # Observation error assignment + - filter: Perform Action + filter variables: + - name: ozoneLayer + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [0.001, 10.1325, 16.00935, 25.43258, 40.32735, 63.93607, 101.325, + 160.0935, 254.3257, 403.2735, 639.3608, 1013.25, 1600.935, 2543.258, 4032.735, + 6393.607, 10132.5, 16009.35, 25432.57, 40327.35, 63936.07, 101325] + errors: [7.7236, 0.020, 0.020, 0.025, 0.080, 0.150, 0.056, 0.125, 0.200, + 0.299, 0.587, 0.864, 1.547, 2.718, 3.893, 4.353, 3.971, 4.407, 4.428, + 3.312, 2.198, 2.285] + # errors: [7.7236, 0.020, 0.020, 0.025, 0.040, 0.080, 0.156, 0.245, 0.510, 1.098, 3.917, 6.124, 6.347, 5.798, 6.843, 9.253,10.091,10.967, 8.478, 5.572, 2.638, 3.525] # operational from gfs.v16.3.9 (late 2023) + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Do not assimilation where pressure is zero + # Zero pressure indicates the data is total column ozone + - filter: RejectList + filter variables: + - name: ozoneLayer + where: + - variable: + name: MetaData/pressure + maxvalue: 0.0001 + + # Sanity check on observaton values + - filter: Bounds Check + filter variables: + - name: ozoneLayer + minvalue: 0 + maxvalue: 1000 + action: + name: reject + + # Total Ozone Quality Check (keeps 0, 2) + # 0 indentifies good data + # 2 identifies good data with a solar zenith angle > 84 degrees + - filter: RejectList + filter variables: + - name: ozoneLayer + where: + - variable: + name: MetaData/totalOzoneQuality + is_not_in: 0, 2 + + # Profile Ozone Quality Check (keeps 0, 1, 7) + # 0 : good data + # 1 : good data with a solar zenith angle > 84 degrees + # 7 : profile for which stray light correction applied + - filter: RejectList + filter variables: + - name: ozoneLayer + where: + - variable: + name: MetaData/profileOzoneQuality + is_not_in: 0, 1, 7 + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Gross error check + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 120 + action: + name: reject + where: + - variable: + name: MetaData/pressure + maxvalue: 0.001 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 30 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 30000.0 + maxvalue: 110000.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 40 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 20000.0 + maxvalue: 30000.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 44.42 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 10100.0 + maxvalue: 20000.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 57.52 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 6400.0 + maxvalue: 10100.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 69.4 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 4000.0 + maxvalue: 6400.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 70 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 2600.0 + maxvalue: 4000.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 62.73 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 1600.0 + maxvalue: 2600.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 50.52 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 1100.0 + maxvalue: 1600.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 35.9 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 700.0 + maxvalue: 1100.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 26.41 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 400.0 + maxvalue: 700.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 20.51 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 300.0 + maxvalue: 400.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 12.82 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 200.0 + maxvalue: 300.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 10 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 70.0 + maxvalue: 200.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 5 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 40.0 + maxvalue: 70.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 2 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 30.0 + maxvalue: 40.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 1 + action: + name: reject + where: + - variable: + name: MetaData/pressure + maxvalue: 30.0 + + # End of Filters + + # Observation Localizations (LocalEnsembleDA) + # ------------------------------------------- + obs localizations: + - localization method: Horizontal Gaspari-Cohn + lengthscale: 1250e3 + max nobs: 10000 + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/ompstc_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/ompstc_npp.yaml.j2 new file mode 100644 index 000000000..b76b56e44 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/ompstc_npp.yaml.j2 @@ -0,0 +1,144 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: ompstc_npp + distribution: + name: Halo + halo size: 1250e3 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [ozoneTotal] + + #obs operator: + # name: AtmVertInterpLay + # geovals: [mole_fraction_of_ozone_in_air] + # coefficients: [0.007886131] # convert from ppmv to DU + # nlevels: [1] + + # Observation Operator + # -------------------- + obs operator: + name: ColumnRetrieval + nlayers_retrieval: 1 + tracer variables: [mole_fraction_of_ozone_in_air] + isApriori: false + isAveragingKernel: false + totalNoVertice: true + stretchVertices: topbottom # options: top, bottom, topbottom, none + model units coeff: 2.241398632746E-3 + + # Observation Pre Filters (QC) + # ---------------------------- + obs pre filters: + - filter: Perform Action + filter variables: + - name: ozoneTotal + action: + name: assign error + error parameter: 6.0 + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # GSI read routine QC + # range sanity check + - filter: Bounds Check + filter variables: + - name: ozoneTotal + minvalue: 0 + maxvalue: 1000 + action: + name: reject + + #- filter: Gaussian Thinning + # horizontal_mesh: 150 + # use_reduced_horizontal_grid: true + # distance_norm: geodesic + # action: + # name: reject + + # Accept total_ozone_error_flag values of 0 and 1, but not any others. + - filter: RejectList + filter variables: + - name: ozoneTotal + where: + - variable: + name: MetaData/totalOzoneQualityCode + is_not_in: 0, 1 + + - filter: RejectList + filter variables: + - name: ozoneTotal + where: + - variable: + name: MetaData/bestOzoneAlgorithmFlag + is_in: 3, 13 + + # GSI setup routine QC + - filter: RejectList + filter variables: + - name: ozoneTotal + where: + - variable: + name: MetaData/sensorScanPosition + is_in: 1, 2, 3, 4, 35 + - variable: + name: MetaData/latitude + minvalue: 50.0 + + - filter: RejectList + filter variables: + - name: ozoneTotal + where: + - variable: + name: MetaData/sensorScanPosition + is_in: 1, 2, 3, 4, 35 + - variable: + name: MetaData/latitude + maxvalue: -50.0 + + - filter: Gaussian Thinning + horizontal_mesh: 150 + use_reduced_horizontal_grid: true + distance_norm: geodesic + action: + name: reject + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + - filter: Background Check + filter variables: + - name: ozoneTotal + threshold: 10.0 + absolute threshold: 300.0 + action: + name: reject + + # End of Filters + + # Observation Localizations (LocalEnsembleDA) + # ------------------------------------------- + obs localizations: + - localization method: Horizontal Gaspari-Cohn + lengthscale: 1250e3 + max nobs: 10000 + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-18.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-18.yaml.j2 new file mode 100644 index 000000000..aa394dfe7 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-18.yaml.j2 @@ -0,0 +1,610 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: satwind_goes-18 + distribution: + name: Halo + halo size: 1250e3 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + # Observation Operator + # -------------------- + obs operator: + name: VertInterp + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: VertInterp + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: false + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Assign the initial observation error, based on height/pressure + # Hard-wiring to prepobs_errtable.global by Type + # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML + # Type 240 (GOES SWIR): Assigned all dummy values in prepobs_errtable.global + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 0.] #Pressure (Pa) + errors: [1000000000., 1000000000.] + # Type 245 (GOES LWIR): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 245 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, + 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., + 14., 14., 14., 14., 14., 14.] + # Type 246 (GOES cloud-top WV): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 246 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, + 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., + 14., 14., 14., 14., 14., 14.] + # Type 247 (GOES clear-sky WV): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 247 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, + 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., + 14., 14., 14., 14., 14., 14.] + # Type 251 (GOES VIS): Assigned all dummy values + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 251 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 0.] #Pressure (Pa) + errors: [1000000000., 1000000000.] + # sanity-check criteria + # Observation Range Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130. + maxvalue: 130. + action: + name: reject + + # Velocity Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130. + action: + name: reject + + # GSI read routine QC (part-1) + # Exclude Type 251 (VIS) --- obs tossed without passing to setup routine + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 251 + action: + name: reject + + # Exclude data with satellite zenith angle > 68 for all types --- obs tossed without passing to setup routine + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: MetaData/satelliteZenithAngle + maxvalue: 68. + action: + name: reject + + # Reject obs with pressure < 12500 pa --- obs tosed without passing to setup routine + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + test variables: + - name: MetaData/pressure + minvalue: 12500. + action: + name: reject + + # Exclude data over non-water surface type where latitude > 20N for Type 240 (IRSW) and Type 245 (IRLW) --- obs tossed and not passed to setup routine + # Notes: This check was missing, so added (eliu) + # Replace land_type_index_NPOSS with water_area_fraction (eliu) + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240, 245 + - variable: MetaData/latitude + minvalue: 20. + test variables: + - name: GeoVaLs/water_area_fraction + minvalue: 0.99 + action: + name: reject + + # GSI read routine QC (part-2) + # Reject obs with qiWithoutForecast < 90. OR > 100. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + test variables: + - name: MetaData/qiWithoutForecast + minvalue: 90. + maxvalue: 100. + action: + name: reject + + # Reject obs with pressure < 15000 pa. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + test variables: + - name: MetaData/pressure + minvalue: 15000. + action: + name: reject + + # Reject obs with pressure < 70000 pa. when Type=251 + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + - variable: ObsType/windEastward + is_in: 251 + test variables: + - name: MetaData/pressure + minvalue: 70000. + action: + name: reject + + # Reject obs with pressure > 30000. when Type=246 + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + - variable: ObsType/windEastward + is_in: 246 + test variables: + - name: MetaData/pressure + maxvalue: 30000. + action: + name: reject + + # Reject obs with pressure > 85000. when isli=1 (land surface) + # Notes: Replace land_type_index_NPOESS with land_area_fraction (eliu) + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + - variable: GeoVaLs/land_area_fraction + minvalue: 0.99 + test variables: + - name: MetaData/pressure + maxvalue: 85000. + action: + name: reject + + # Reject obs with pct1 (Coeff. of Var.) outside of 0.04–0.5, Type [240,245,246,251] ONLY + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + - variable: ObsType/windEastward + is_in: 240, 245, 246, 251 + test variables: + - name: MetaData/coefficientOfVariation + minvalue: 0.04 + maxvalue: 0.5 + action: + name: reject + + # NESDIS obs are also subject to the experr_norm test defined as: + # + # if (10. - 0.1*(expectedError))/(ob_speed)>0.9, or ob_speed<0.1, reject, applies to NESDIS winds + # + # CLEARED: With caveat that float precision/handling differences can generate different acceptance criteria + # between UFO and GSI for observations with an experr_norm value right around the maxvalue. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + test variables: + - name: ObsFunction/SatWindsErrnormCheck + maxvalue: 0.9 + action: + name: reject + + # Reject all Type=240 (GOES SWIR) AMVs: These are not currently assimilated in GSI and they have missing-values + # assigned to ob-errors + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240 + action: + name: reject + + # GSI setupw routine QC + # Reject any ob Type [240–260] when pressure greater than 950 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240-260 + test variables: + - name: MetaData/pressure + maxvalue: 95001. + action: + name: reject + + # GOES IR (245) reject when pressure between 399 and 801 mb. + # # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/pressure + minvalue: 39901. + maxvalue: 80099. + - variable: ObsType/windEastward + is_in: 245 + action: + name: reject + + # GOES WV (246, 250, 254), reject when pressure greater than 399 mb. + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 246, 250, 254 + test variables: + - name: MetaData/pressure + maxvalue: 39900. + action: + name: reject + + # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation + # Turn this check off for now. + # Need to check if troposphere pressure was implemented correctly in fv3-jed + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + reference: GeoVaLs/tropopause_pressure + value: MetaData/pressure + minvalue: -5000. # 50 hPa above tropopause level, negative p-diff + action: + name: reject + + # GOES (247) reject any observation with a /=0 surface type (non-water + # surface) within 110 hPa of the surface pressure (as part of the LNVD # check). + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + # Notes (eliu): Replace land_type_index_NPOESS with land_area_fraction. + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + - variable: + name: ObsType/windEastward + is_in: 247 + reference: GeoVaLs/surface_pressure + value: MetaData/pressure + maxvalue: -11000. # within 110 hPa above surface pressure, negative p-diff + action: + name: reject + + # Reject GOES (247) when difference of wind direction is more than 50 degrees. + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 247 + test variables: + - name: ObsFunction/WindDirAngleDiff + maxvalue: 50. + action: + name: reject + + # All satwinds must adjust errors based on ObsErrorFactorPressureCheck + # prior to the SPDB check (i.e. the gross-error check). The gross-error + # check uses the adjusted errors for error-bound tightening and rejection, + # so this check has to come first. This check will inflate errors for obs + # that are too close to either the model top or bottom. + # Notes (eliu): GMAO added a required parameter: adjusted_error_name. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + surface_obs: false + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All satwinds subject to a gross-error check that contains significant + # modifiers for satwinds with a negative speed-bias. ALL wind gross-error + # checks are currently being done by the SatWindsSPDBCheck. + # CLEARED + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, + 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, + 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, + 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, + 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, + 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, + 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, + 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, + 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don"t want to inflate errors for duplication if one + # of the duplicates should be rejected. + # Notes (eliu): ObsErrorFactorDuplicateCheck obsfunction requires PreUseFlag (usage parameter from read_satwnd.f90). + # : Turn off duplicate check for now. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + # We are extending this to an additional filter that inflates final ob-errors across-the-board by + # 1/0.8 = 1.25. This is caused by the GSI value of nvqc being set to .true. in the global operational + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this nvqc functionality were to be switched off (i.e. if variational qc were to be turned off), + # you would want to remove this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + # End of Filters + + # Observation Localizations (LocalEnsembleDA) + # ------------------------------------------- + obs localizations: + - localization method: Horizontal Gaspari-Cohn + lengthscale: 1250e3 + max nobs: 10000 + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.leogeo_multi.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.leogeo_multi.yaml.j2 new file mode 100644 index 000000000..c0dd03dd0 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.leogeo_multi.yaml.j2 @@ -0,0 +1,270 @@ +- obs space: + name: satwind_leogeo_multi + distribution: + name: Halo + halo size: 1250e3 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + obs operator: + name: VertInterp + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + linear obs operator: + name: VertInterp + + # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI's thinning of AHI/Himawari-8 satwinds + # results in more JEDI satwinds in the diag file than in GSI, but far fewer JEDI satwinds assimilated than + # GSI. JEDI under-counts assimilated winds by roughly 25-40%, relative to GSI, and this under-count is not + # even including the temporal thinning which is applied in GSI but not JEDI (by this filter below). See + # GDASApp Issue #741 for details: https://github.com/NOAA-EMC/GDASApp/issues/741 + #obs pre filters: + #- filter: Gaussian Thinning + # horizontal_mesh: 200 + # vertical_mesh: 10000 + # use_reduced_horizontal_grid: true + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: False + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + + obs post filters: + # Assign the initial observation error, based on height/pressure + # Hard-wiring to prepobs_errtable.global by Type + # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML + + # Type 255 (LEOGEO LWIR) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 255 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000.,105000.,100000.,95000.,90000.,85000.,80000.,75000.,70000.,65000.,60000.,55000.,50000.,45000.,40000.,35000.,30000.,25000.,20000.,15000.,10000.,7500.,5000.,4000.,3000.,2000.,1000.,500.,400.,300.,200.,100.,0.] #Pressure (Pa) + errors: [3.8,3.8,3.8,3.8,3.8,3.8,3.8,3.8,3.9,3.9,4.,4.,4.1,5.,6.,6.3,6.6,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.] + + # sanity-check criteria + # Observation Range Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130. + maxvalue: 130. + action: + name: reject + + # Velocity Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130. + action: + name: reject + + # GSI setupw routine QC + # Reject any ob Type [240–260] when pressure greater than 950 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240-260 + test variables: + - name: MetaData/pressure + maxvalue: 95001. + action: + name: reject + + # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation + # Turn this check off for now. + # Need to check if troposphere pressure was implemented correctly in fv3-jed + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + reference: GeoVaLs/tropopause_pressure + value: MetaData/pressure + minvalue: -5000. # 50 hPa above tropopause level, negative p-diff + action: + name: reject + + # All satwinds must adjust errors based on ObsErrorFactorPressureCheck + # prior to the SPDB check (i.e. the gross-error check). The gross-error + # check uses the adjusted errors for error-bound tightening and rejection, + # so this check has to come first. This check will inflate errors for obs + # that are too close to either the model top or bottom. + # Notes (eliu): GMAO added a required parameter: adjusted_error_name. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + surface_obs: false + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All satwinds subject to a gross-error check that contains significant + # modifiers for satwinds with a negative speed-bias. ALL wind gross-error + # checks are currently being done by the SatWindsSPDBCheck. + # CLEARED + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [ 2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [ 2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don't want to inflate errors for duplication if one + # of the duplicates should be rejected. + # Notes (eliu): ObsErrorFactorDuplicateCheck obsfunction requires PreUseFlag (usage parameter from read_satwnd.f90). + # : Turn off duplicate check for now. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + + # We are extending this to an additional filter that inflates final ob-errors across-the-board by + # 1/0.8 = 1.25. This is caused by the GSI value of nvqc being set to .true. in the global operational + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this nvqc functionality were to be switched off (i.e. if variational qc were to be turned off), + # you would want to remove this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + + # End of Filters + + # Observation Localizations (LocalEnsembleDA) + # ------------------------------------------- + obs localizations: + - localization method: Horizontal Gaspari-Cohn + lengthscale: 1250e3 + max nobs: 10000 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.viirs_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.viirs_n20.yaml.j2 new file mode 100644 index 000000000..b3a4abb42 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.viirs_n20.yaml.j2 @@ -0,0 +1,298 @@ +- obs space: + name: satwind_viirs_n20 + distribution: + name: Halo + halo size: 1250e3 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + obs operator: + name: VertInterp + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + linear obs operator: + name: VertInterp + + # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI's thinning of AHI/Himawari-8 satwinds + # results in more JEDI satwinds in the diag file than in GSI, but far fewer JEDI satwinds assimilated than + # GSI. JEDI under-counts assimilated winds by roughly 25-40%, relative to GSI, and this under-count is not + # even including the temporal thinning which is applied in GSI but not JEDI (by this filter below). See + # GDASApp Issue #741 for details: https://github.com/NOAA-EMC/GDASApp/issues/741 + #obs pre filters: + #- filter: Gaussian Thinning + # horizontal_mesh: 200 + # vertical_mesh: 10000 + # use_reduced_horizontal_grid: true + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: False + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + + obs post filters: + # Assign the initial observation error, based on height/pressure + # Hard-wiring to prepobs_errtable.global by Type + # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML + + # Type 260 (VIIRS LWIR) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 260 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000.,105000.,100000.,95000.,90000.,85000.,80000.,75000.,70000.,65000.,60000.,55000.,50000.,45000.,40000.,35000.,30000.,25000.,20000.,15000.,10000.,7500.,5000.,4000.,3000.,2000.,1000.,500.,400.,300.,200.,100.,0.] + errors: [3.8,3.8,3.8,3.8,3.8,3.8,3.8,3.8,3.9,3.9,4.,4.,4.1,5.,6.,6.3,6.6,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.] + + # sanity-check criteria + # Observation Range Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130. + maxvalue: 130. + action: + name: reject + + # Velocity Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130. + action: + name: reject + + # Reject any observation with a /=0 surface type (non-water surface) within + # 200 hPa of the surface pressure (as part of the LNVD check). + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + reference: GeoVaLs/surface_pressure + value: MetaData/pressure + maxvalue: -20000. # within 200 hPa above surface pressure, negative p-diff + action: + name: reject + + # LNVD check + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/SatWindsLNVDCheck + maxvalue: 3. + action: + name: reject + + # GSI setupw routine QC + # Reject any ob Type [240–260] when pressure greater than 950 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240-260 + test variables: + - name: MetaData/pressure + maxvalue: 95001. + action: + name: reject + + # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation + # Turn this check off for now. + # Need to check if troposphere pressure was implemented correctly in fv3-jed + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + reference: GeoVaLs/tropopause_pressure + value: MetaData/pressure + minvalue: -5000. # 50 hPa above tropopause level, negative p-diff + action: + name: reject + + # All satwinds must adjust errors based on ObsErrorFactorPressureCheck + # prior to the SPDB check (i.e. the gross-error check). The gross-error + # check uses the adjusted errors for error-bound tightening and rejection, + # so this check has to come first. This check will inflate errors for obs + # that are too close to either the model top or bottom. + # Notes (eliu): GMAO added a required parameter: adjusted_error_name. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + surface_obs: false + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All satwinds subject to a gross-error check that contains significant + # modifiers for satwinds with a negative speed-bias. ALL wind gross-error + # checks are currently being done by the SatWindsSPDBCheck. + # CLEARED + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [ 2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [ 2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don't want to inflate errors for duplication if one + # of the duplicates should be rejected. + # Notes (eliu): ObsErrorFactorDuplicateCheck obsfunction requires PreUseFlag (usage parameter from read_satwnd.f90). + # : Turn off duplicate check for now. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + + # We are extending this to an additional filter that inflates final ob-errors across-the-board by + # 1/0.8 = 1.25. This is caused by the GSI value of nvqc being set to .true. in the global operational + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this nvqc functionality were to be switched off (i.e. if variational qc were to be turned off), + # you would want to remove this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + + # End of Filters + + # ------------------------------------------- + obs localizations: + - localization method: Horizontal Gaspari-Cohn + lengthscale: 1250e3 + max nobs: 10000 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.viirs_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.viirs_npp.yaml.j2 new file mode 100644 index 000000000..851cad1ed --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.viirs_npp.yaml.j2 @@ -0,0 +1,298 @@ +- obs space: + name: satwind_viirs_npp + distribution: + name: Halo + halo size: 1250e3 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + obs operator: + name: VertInterp + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + linear obs operator: + name: VertInterp + + # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI's thinning of AHI/Himawari-8 satwinds + # results in more JEDI satwinds in the diag file than in GSI, but far fewer JEDI satwinds assimilated than + # GSI. JEDI under-counts assimilated winds by roughly 25-40%, relative to GSI, and this under-count is not + # even including the temporal thinning which is applied in GSI but not JEDI (by this filter below). See + # GDASApp Issue #741 for details: https://github.com/NOAA-EMC/GDASApp/issues/741 + #obs pre filters: + #- filter: Gaussian Thinning + # horizontal_mesh: 200 + # vertical_mesh: 10000 + # use_reduced_horizontal_grid: true + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: False + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + + obs post filters: + # Assign the initial observation error, based on height/pressure + # Hard-wiring to prepobs_errtable.global by Type + # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML + + # Type 260 (VIIRS LWIR) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 260 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000.,105000.,100000.,95000.,90000.,85000.,80000.,75000.,70000.,65000.,60000.,55000.,50000.,45000.,40000.,35000.,30000.,25000.,20000.,15000.,10000.,7500.,5000.,4000.,3000.,2000.,1000.,500.,400.,300.,200.,100.,0.] + errors: [3.8,3.8,3.8,3.8,3.8,3.8,3.8,3.8,3.9,3.9,4.,4.,4.1,5.,6.,6.3,6.6,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.] + + # sanity-check criteria + # Observation Range Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130. + maxvalue: 130. + action: + name: reject + + # Velocity Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130. + action: + name: reject + + # Reject any observation with a /=0 surface type (non-water surface) within + # 200 hPa of the surface pressure (as part of the LNVD check). + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + reference: GeoVaLs/surface_pressure + value: MetaData/pressure + maxvalue: -20000. # within 200 hPa above surface pressure, negative p-diff + action: + name: reject + + # LNVD check + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/SatWindsLNVDCheck + maxvalue: 3. + action: + name: reject + + # GSI setupw routine QC + # Reject any ob Type [240–260] when pressure greater than 950 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240-260 + test variables: + - name: MetaData/pressure + maxvalue: 95001. + action: + name: reject + + # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation + # Turn this check off for now. + # Need to check if troposphere pressure was implemented correctly in fv3-jed + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + reference: GeoVaLs/tropopause_pressure + value: MetaData/pressure + minvalue: -5000. # 50 hPa above tropopause level, negative p-diff + action: + name: reject + + # All satwinds must adjust errors based on ObsErrorFactorPressureCheck + # prior to the SPDB check (i.e. the gross-error check). The gross-error + # check uses the adjusted errors for error-bound tightening and rejection, + # so this check has to come first. This check will inflate errors for obs + # that are too close to either the model top or bottom. + # Notes (eliu): GMAO added a required parameter: adjusted_error_name. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + surface_obs: false + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All satwinds subject to a gross-error check that contains significant + # modifiers for satwinds with a negative speed-bias. ALL wind gross-error + # checks are currently being done by the SatWindsSPDBCheck. + # CLEARED + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [ 2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [ 2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don't want to inflate errors for duplication if one + # of the duplicates should be rejected. + # Notes (eliu): ObsErrorFactorDuplicateCheck obsfunction requires PreUseFlag (usage parameter from read_satwnd.f90). + # : Turn off duplicate check for now. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + + # We are extending this to an additional filter that inflates final ob-errors across-the-board by + # 1/0.8 = 1.25. This is caused by the GSI value of nvqc being set to .true. in the global operational + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this nvqc functionality were to be switched off (i.e. if variational qc were to be turned off), + # you would want to remove this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + + # End of Filters + +# ------------------------------------------- + obs localizations: + - localization method: Horizontal Gaspari-Cohn + lengthscale: 1250e3 + max nobs: 10000 diff --git a/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 index 839821963..c16a22c51 100644 --- a/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 @@ -40,19 +40,19 @@ variational bc: predictors: - name: constant - - name: lapse_rate + - name: lapseRate order: 2 tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" - - name: lapse_rate + - name: lapseRate tlapse: *{{observation_from_jcb}}_tlapse - - name: emissivity - - name: scan_angle + - name: emissivityJacobian + - name: sensorScanAngle order: 4 - - name: scan_angle + - name: sensorScanAngle order: 3 - - name: scan_angle + - name: sensorScanAngle order: 2 - - name: scan_angle + - name: sensorScanAngle covariance: minimal required obs number: 20 variance range: [1.0e-6, 10.0] diff --git a/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 index 7c68b3e89..d18985cd8 100644 --- a/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 @@ -40,19 +40,19 @@ variational bc: predictors: - name: constant - - name: lapse_rate + - name: lapseRate order: 2 tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" - - name: lapse_rate + - name: lapseRate tlapse: *{{observation_from_jcb}}_tlapse - - name: emissivity - - name: scan_angle + - name: emissivityJacobian + - name: sensorScanAngle order: 4 - - name: scan_angle + - name: sensorScanAngle order: 3 - - name: scan_angle + - name: sensorScanAngle order: 2 - - name: scan_angle + - name: sensorScanAngle covariance: minimal required obs number: 20 variance range: [1.0e-6, 10.0] diff --git a/parm/jcb-gdas/observations/atmosphere/conv_ps.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/conv_ps.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 index 9e1fb87cf..6bea2e3cc 100644 --- a/parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 @@ -37,19 +37,19 @@ variational bc: predictors: - name: constant - - name: lapse_rate + - name: lapseRate order: 2 tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" - - name: lapse_rate + - name: lapseRate tlapse: *{{observation_from_jcb}}_tlapse - - name: emissivity - - name: scan_angle + - name: emissivityJacobian + - name: sensorScanAngle order: 4 - - name: scan_angle + - name: sensorScanAngle order: 3 - - name: scan_angle + - name: sensorScanAngle order: 2 - - name: scan_angle + - name: sensorScanAngle covariance: minimal required obs number: 20 variance range: [1.0e-6, 10.0] diff --git a/parm/jcb-gdas/observations/atmosphere/cris-fsr_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/cris-fsr_npp.yaml.j2 index be21f7863..5e3cae3b7 100644 --- a/parm/jcb-gdas/observations/atmosphere/cris-fsr_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/cris-fsr_npp.yaml.j2 @@ -37,19 +37,19 @@ variational bc: predictors: - name: constant - - name: lapse_rate + - name: lapseRate order: 2 tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" - - name: lapse_rate + - name: lapseRate tlapse: *{{observation_from_jcb}}_tlapse - - name: emissivity - - name: scan_angle + - name: emissivityJacobian + - name: sensorScanAngle order: 4 - - name: scan_angle + - name: sensorScanAngle order: 3 - - name: scan_angle + - name: sensorScanAngle order: 2 - - name: scan_angle + - name: sensorScanAngle covariance: minimal required obs number: 20 variance range: [1.0e-6, 10.0] diff --git a/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/mtiasi_metop-a.yaml.j2 similarity index 99% rename from parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/mtiasi_metop-a.yaml.j2 index 141ddfc0a..8e1ae98b4 100644 --- a/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/mtiasi_metop-a.yaml.j2 @@ -39,19 +39,19 @@ variational bc: predictors: - name: constant - - name: lapse_rate + - name: lapseRate order: 2 tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" - - name: lapse_rate + - name: lapseRate tlapse: *{{observation_from_jcb}}_tlapse - - name: emissivity - - name: scan_angle + - name: emissivityJacobian + - name: sensorScanAngle order: 4 - - name: scan_angle + - name: sensorScanAngle order: 3 - - name: scan_angle + - name: sensorScanAngle order: 2 - - name: scan_angle + - name: sensorScanAngle covariance: minimal required obs number: 20 variance range: [1.0e-6, 10.0] diff --git a/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/mtiasi_metop-b.yaml.j2 similarity index 99% rename from parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/mtiasi_metop-b.yaml.j2 index 051d62109..dd1cff322 100644 --- a/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/mtiasi_metop-b.yaml.j2 @@ -39,19 +39,19 @@ variational bc: predictors: - name: constant - - name: lapse_rate + - name: lapseRate order: 2 tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" - - name: lapse_rate + - name: lapseRate tlapse: *{{observation_from_jcb}}_tlapse - - name: emissivity - - name: scan_angle + - name: emissivityJacobian + - name: sensorScanAngle order: 4 - - name: scan_angle + - name: sensorScanAngle order: 3 - - name: scan_angle + - name: sensorScanAngle order: 2 - - name: scan_angle + - name: sensorScanAngle covariance: minimal required obs number: 20 variance range: [1.0e-6, 10.0] diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-18.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-18.yaml.j2 new file mode 100644 index 000000000..f1e61fbed --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-18.yaml.j2 @@ -0,0 +1,601 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: satwind_goes-18 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + # Observation Operator + # -------------------- + obs operator: + name: VertInterp + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: VertInterp + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: false + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Assign the initial observation error, based on height/pressure + # Hard-wiring to prepobs_errtable.global by Type + # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML + # Type 240 (GOES SWIR): Assigned all dummy values in prepobs_errtable.global + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 0.] #Pressure (Pa) + errors: [1000000000., 1000000000.] + # Type 245 (GOES LWIR): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 245 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, + 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., + 14., 14., 14., 14., 14., 14.] + # Type 246 (GOES cloud-top WV): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 246 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, + 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., + 14., 14., 14., 14., 14., 14.] + # Type 247 (GOES clear-sky WV): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 247 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, + 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., + 14., 14., 14., 14., 14., 14.] + # Type 251 (GOES VIS): Assigned all dummy values + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 251 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 0.] #Pressure (Pa) + errors: [1000000000., 1000000000.] + # sanity-check criteria + # Observation Range Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130. + maxvalue: 130. + action: + name: reject + + # Velocity Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130. + action: + name: reject + + # GSI read routine QC (part-1) + # Exclude Type 251 (VIS) --- obs tossed without passing to setup routine + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 251 + action: + name: reject + + # Exclude data with satellite zenith angle > 68 for all types --- obs tossed without passing to setup routine + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: MetaData/satelliteZenithAngle + maxvalue: 68. + action: + name: reject + + # Reject obs with pressure < 12500 pa --- obs tosed without passing to setup routine + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + test variables: + - name: MetaData/pressure + minvalue: 12500. + action: + name: reject + + # Exclude data over non-water surface type where latitude > 20N for Type 240 (IRSW) and Type 245 (IRLW) --- obs tossed and not passed to setup routine + # Notes: This check was missing, so added (eliu) + # Replace land_type_index_NPOSS with water_area_fraction (eliu) + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240, 245 + - variable: MetaData/latitude + minvalue: 20. + test variables: + - name: GeoVaLs/water_area_fraction + minvalue: 0.99 + action: + name: reject + + # GSI read routine QC (part-2) + # Reject obs with qiWithoutForecast < 90. OR > 100. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + test variables: + - name: MetaData/qiWithoutForecast + minvalue: 90. + maxvalue: 100. + action: + name: reject + + # Reject obs with pressure < 15000 pa. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + test variables: + - name: MetaData/pressure + minvalue: 15000. + action: + name: reject + + # Reject obs with pressure < 70000 pa. when Type=251 + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + - variable: ObsType/windEastward + is_in: 251 + test variables: + - name: MetaData/pressure + minvalue: 70000. + action: + name: reject + + # Reject obs with pressure > 30000. when Type=246 + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + - variable: ObsType/windEastward + is_in: 246 + test variables: + - name: MetaData/pressure + maxvalue: 30000. + action: + name: reject + + # Reject obs with pressure > 85000. when isli=1 (land surface) + # Notes: Replace land_type_index_NPOESS with land_area_fraction (eliu) + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + - variable: GeoVaLs/land_area_fraction + minvalue: 0.99 + test variables: + - name: MetaData/pressure + maxvalue: 85000. + action: + name: reject + + # Reject obs with pct1 (Coeff. of Var.) outside of 0.04–0.5, Type [240,245,246,251] ONLY + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + - variable: ObsType/windEastward + is_in: 240, 245, 246, 251 + test variables: + - name: MetaData/coefficientOfVariation + minvalue: 0.04 + maxvalue: 0.5 + action: + name: reject + + # NESDIS obs are also subject to the experr_norm test defined as: + # + # if (10. - 0.1*(expectedError))/(ob_speed)>0.9, or ob_speed<0.1, reject, applies to NESDIS winds + # + # CLEARED: With caveat that float precision/handling differences can generate different acceptance criteria + # between UFO and GSI for observations with an experr_norm value right around the maxvalue. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/satelliteIdentifier + is_in: 250-299 + test variables: + - name: ObsFunction/SatWindsErrnormCheck + maxvalue: 0.9 + action: + name: reject + + # Reject all Type=240 (GOES SWIR) AMVs: These are not currently assimilated in GSI and they have missing-values + # assigned to ob-errors + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240 + action: + name: reject + + # GSI setupw routine QC + # Reject any ob Type [240–260] when pressure greater than 950 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240-260 + test variables: + - name: MetaData/pressure + maxvalue: 95001. + action: + name: reject + + # GOES IR (245) reject when pressure between 399 and 801 mb. + # # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/pressure + minvalue: 39901. + maxvalue: 80099. + - variable: ObsType/windEastward + is_in: 245 + action: + name: reject + + # GOES WV (246, 250, 254), reject when pressure greater than 399 mb. + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 246, 250, 254 + test variables: + - name: MetaData/pressure + maxvalue: 39900. + action: + name: reject + + # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation + # Turn this check off for now. + # Need to check if troposphere pressure was implemented correctly in fv3-jed + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + reference: GeoVaLs/tropopause_pressure + value: MetaData/pressure + minvalue: -5000. # 50 hPa above tropopause level, negative p-diff + action: + name: reject + + # GOES (247) reject any observation with a /=0 surface type (non-water + # surface) within 110 hPa of the surface pressure (as part of the LNVD # check). + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + # Notes (eliu): Replace land_type_index_NPOESS with land_area_fraction. + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + - variable: + name: ObsType/windEastward + is_in: 247 + reference: GeoVaLs/surface_pressure + value: MetaData/pressure + maxvalue: -11000. # within 110 hPa above surface pressure, negative p-diff + action: + name: reject + + # Reject GOES (247) when difference of wind direction is more than 50 degrees. + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 247 + test variables: + - name: ObsFunction/WindDirAngleDiff + maxvalue: 50. + action: + name: reject + + # All satwinds must adjust errors based on ObsErrorFactorPressureCheck + # prior to the SPDB check (i.e. the gross-error check). The gross-error + # check uses the adjusted errors for error-bound tightening and rejection, + # so this check has to come first. This check will inflate errors for obs + # that are too close to either the model top or bottom. + # Notes (eliu): GMAO added a required parameter: adjusted_error_name. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + surface_obs: false + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All satwinds subject to a gross-error check that contains significant + # modifiers for satwinds with a negative speed-bias. ALL wind gross-error + # checks are currently being done by the SatWindsSPDBCheck. + # CLEARED + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, + 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, + 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, + 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, + 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, + 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, + 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, + 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, + 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don"t want to inflate errors for duplication if one + # of the duplicates should be rejected. + # Notes (eliu): ObsErrorFactorDuplicateCheck obsfunction requires PreUseFlag (usage parameter from read_satwnd.f90). + # : Turn off duplicate check for now. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + # We are extending this to an additional filter that inflates final ob-errors across-the-board by + # 1/0.8 = 1.25. This is caused by the GSI value of nvqc being set to .true. in the global operational + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this nvqc functionality were to be switched off (i.e. if variational qc were to be turned off), + # you would want to remove this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + # End of Filters + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 index 763f27467..ef47275c6 100644 --- a/parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 @@ -44,22 +44,22 @@ ch91h: 18 - name: cosine_of_latitude_times_orbit_node - name: sine_of_latitude - - name: lapse_rate + - name: lapseRate order: 2 tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" - - name: lapse_rate + - name: lapseRate tlapse: *{{observation_from_jcb}}_tlapse - - name: emissivity - - name: scan_angle + - name: emissivityJacobian + - name: sensorScanAngle var_name: scan_position order: 4 - - name: scan_angle + - name: sensorScanAngle var_name: scan_position order: 3 - - name: scan_angle + - name: sensorScanAngle var_name: scan_position order: 2 - - name: scan_angle + - name: sensorScanAngle var_name: scan_position # Observation Prior Filters (QC) diff --git a/parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 index 4f55848a3..feffb1e84 100644 --- a/parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 @@ -44,22 +44,22 @@ ch91h: 18 - name: cosine_of_latitude_times_orbit_node - name: sine_of_latitude - - name: lapse_rate + - name: lapseRate order: 2 tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" - - name: lapse_rate + - name: lapseRate tlapse: *{{observation_from_jcb}}_tlapse - - name: emissivity - - name: scan_angle + - name: emissivityJacobian + - name: sensorScanAngle var_name: scan_position order: 4 - - name: scan_angle + - name: sensorScanAngle var_name: scan_position order: 3 - - name: scan_angle + - name: sensorScanAngle var_name: scan_position order: 2 - - name: scan_angle + - name: sensorScanAngle var_name: scan_position # Observation Prior Filters (QC) From 48b28caea44b3cc806071a76e4417693a0542684 Mon Sep 17 00:00:00 2001 From: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com> Date: Fri, 16 Aug 2024 12:38:52 -0400 Subject: [PATCH 047/199] replace fixed distribution name with templated variable (#23) --- .../atmosphere-lgetkf/aircraft.yaml.j2 | 2 +- .../atmosphere-lgetkf/amsua_n19.yaml.j2 | 2 +- .../ascatw.ascat_metop-b.yaml.j2 | 2 +- .../atmosphere-lgetkf/atms_n20.yaml.j2 | 2 +- .../atmosphere-lgetkf/atms_npp.yaml.j2 | 2 +- .../atmosphere-lgetkf/conv_ps.yaml.j2 | 373 ------------------ .../atmosphere-lgetkf/conventional_ps.yaml.j2 | 2 +- .../atmosphere-lgetkf/gnssro.yaml.j2 | 2 +- .../atmosphere-lgetkf/mtiasi_metop-a.yaml.j2 | 2 +- .../atmosphere-lgetkf/mtiasi_metop-b.yaml.j2 | 2 +- .../atmosphere-lgetkf/ompsnp_npp.yaml.j2 | 2 +- .../atmosphere-lgetkf/ompstc_npp.yaml.j2 | 2 +- .../satwnd.abi_goes-16.yaml.j2 | 2 +- .../satwnd.abi_goes-18.yaml.j2 | 2 +- .../atmosphere-lgetkf/satwnd.ahi_h8.yaml.j2 | 2 +- .../satwnd.leogeo_multi.yaml.j2 | 2 +- .../satwnd.viirs_n20.yaml.j2 | 2 +- .../satwnd.viirs_npp.yaml.j2 | 2 +- .../atmosphere-lgetkf/sondes.yaml.j2 | 2 +- 19 files changed, 18 insertions(+), 391 deletions(-) delete mode 100644 parm/jcb-gdas/observations/atmosphere-lgetkf/conv_ps.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml.j2 index 4d4a72ec7..a99271332 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml.j2 @@ -5,7 +5,7 @@ obs space: name: Aircraft distribution: - name: Halo + name: "{{distribution_type}}" halo size: 1250e3 obsdatain: engine: diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 index 15f4846da..17ed934c7 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 @@ -5,7 +5,7 @@ obs space: name: AMSUA N19 distribution: - name: Halo + name: "{{distribution_type}}" halo size: 1250e3 obsdatain: engine: diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/ascatw.ascat_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/ascatw.ascat_metop-b.yaml.j2 index 788ead9ed..35a8a06b7 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/ascatw.ascat_metop-b.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/ascatw.ascat_metop-b.yaml.j2 @@ -5,7 +5,7 @@ obs space: name: ascatw_ascat_metop-b distribution: - name: Halo + name: "{{distribution_type}}" halo size: 1250e3 obsdatain: engine: diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/atms_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/atms_n20.yaml.j2 index 7bfa5e1c2..6c5754f4c 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/atms_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/atms_n20.yaml.j2 @@ -5,7 +5,7 @@ obs space: name: ATMS N20 distribution: - name: Halo + name: "{{distribution_type}}" halo size: 1250e3 obsdatain: engine: diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/atms_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/atms_npp.yaml.j2 index 6356796c5..ea17f1241 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/atms_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/atms_npp.yaml.j2 @@ -5,7 +5,7 @@ obs space: name: ATMS NPP distribution: - name: Halo + name: "{{distribution_type}}" halo size: 1250e3 obsdatain: engine: diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/conv_ps.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/conv_ps.yaml.j2 deleted file mode 100644 index 0ca33b1f2..000000000 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/conv_ps.yaml.j2 +++ /dev/null @@ -1,373 +0,0 @@ -- - - # Observation Space (I/O) - # ----------------------- - obs space: - name: surface_ps - distribution: - name: Halo - halo size: 1250e3 - obsdatain: - engine: - type: H5File - obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" - obsdataout: - engine: - type: H5File - obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" - io pool: - max pool size: 1 - simulated variables: [stationPressure] - - # Observation Operator - # -------------------- - obs operator: - name: SfcPCorrected - variables: - - name: stationPressure - da_psfc_scheme: GSI - station_altitude: height - geovar_sfc_geomz: surface_altitude - geovar_geomz: geopotential_height - - # Linear Observation Operator - # --------------------------- - linear obs operator: - name: Identity - variables: - - name: stationPressure - - # Observation Prior Filters (QC) - # ------------------------------ - obs prior filters: - # Initial Error Assignments for SFC Observations - - filter: Perform Action - filter variables: - - name: stationPressure - where: - - variable: ObsType/stationPressure - is_in: [181] - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - round_to_the_nearest_integer: true - xvar: - name: ObsValue/stationPressure - xvals: [80000, 75000, 70000, 65000, 60000, 55000] - errors: [110, 120, 120, 120, 120, 1.0e+11] - - filter: Perform Action - filter variables: - - name: stationPressure - where: - - variable: ObsType/stationPressure - is_in: [187] - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - round_to_the_nearest_integer: true - xvar: - name: ObsValue/stationPressure - xvals: [85000, 80000, 75000, 70000, 65000, 60000, 55000] - errors: [120, 140, 140, 140, 140, 140, 1.0e+11] - # Initial Error Assignments for SFCSHIP Observations - - filter: Perform Action - filter variables: - - name: stationPressure - where: - - variable: ObsType/stationPressure - is_in: [180] - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - round_to_the_nearest_integer: true - xvar: - name: ObsValue/stationPressure - xvals: [60000, 55000] - errors: [130, 1.0e+11] - - filter: Perform Action - filter variables: - - name: stationPressure - where: - - variable: ObsType/stationPressure - is_in: [183] - action: - name: assign error - error parameter: 1.0e+11 - - # Initial Error Assignments for Radiosonde - - filter: Perform Action - filter variables: - - name: stationPressure - where: - - variable: ObsType/stationPressure - is_in: [120] - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - round_to_the_nearest_integer: true - xvar: - name: ObsValue/stationPressure - xvals: [80000, 75000, 70000, 65000, 60000, 55000] - errors: [110, 120, 120, 120, 120, 1.0e+11] - # Create PreQC group variable (pqm in GSI read_prepbufr) - - filter: Variable Assignment - assignments: - - name: InputObsError/stationPressure - type: float - source variable: ObsErrorData/stationPressure - - # Set observation quality-realted variables - # Create PreQC group variable (pqm in GSI read_prepbufr) - - filter: Variable Assignment - assignments: - - name: PreQC/stationPressure - type: int - source variable: QualityMarker/stationPressure - - # Create PreUseFlag group variable (usage in GSI read_prepbufr) - # Initialize - - filter: Variable Assignment - assignments: - - name: PreUseFlag/stationPressure - type: int - source variable: PreQC/stationPressure - - - filter: Variable Assignment - where: - - variable: - name: PreUseFlag/stationPressure - is_in: 1-15 - assignments: - - name: PreUseFlag/stationPressure - value: 0 - # Re-assignment - - filter: Variable Assignment - where: - - variable: - name: ObsType/stationPressure - is_in: 183 - assignments: - - name: PreUseFlag/stationPressure - value: 100 - - - filter: Variable Assignment - where: - - variable: - name: ObsValue/stationPressure - is_defined: - - variable: - name: ObsValue/stationPressure - maxvalue: 50000.00 - where operator: and - assignments: - - name: PreUseFlag/stationPressure - value: 100 - - - filter: Variable Assignment - where: - - variable: - name: PreQC/stationPressure - is_in: 9, 12, 15 - assignments: - - name: PreUseFlag/stationPressure - value: 100 - - - filter: Variable Assignment - where: - - variable: - name: PreQC/stationPressure - is_in: 4-15 - assignments: - - name: PreUseFlag/stationPressure - value: 101 - - ## Observation range sanity check - #- filter: Bounds Check - # filter variables: - # - name: stationPressure - # minvalue: 37499.0 - # maxvalue: 106999.0 - # action: - # name: reject - ## Reject all ObsType 183 - #- filter: RejectList - # where: - # - variable: - # name: ObsType/stationPressure - # is_in: 183 - ## Reject surface pressure below 500 hPa - #- filter: Bounds Check - # filter variables: - # - name: stationPressure - # minvalue: 50000.00 - # action: - # name: reject - #- filter: RejectList - # where: - # - variable: - # name: PreQC/stationPressure - # is_in: 4-15 - # Inflate obs error based on obs type - - filter: Perform Action - filter variables: - - name: stationPressure - where: - - variable: PreQC/stationPressure - is_in: 3, 7 - action: - name: inflate error - inflation factor: 1.2 - - # Observation Post Filters (QC) - # ----------------------------- - obs post filters: - # Calculate obs error inflation factors for duplicated observations at the same location - - filter: Variable Assignment - assignments: - - name: ObsErrorFactorDuplicateCheck/stationPressure - type: float - function: - name: ObsFunction/ObsErrorFactorDuplicateCheck - options: - use_air_pressure: false - variable: stationPressure - - # Reduce effective observation error based on obs type and subtype - # In this case: reduce effective obs error for buoy - - filter: Perform Action - filter variables: - - name: stationPressure - where: - - variable: - name: ObsType/stationPressure - is_in: 180 - - variable: - name: ObsSubType/stationPressure - is_in: 0 - action: - name: inflate error - inflation factor: 0.7 - - # Calculate obs error inflation factors for large discrepancies between model and observations - - filter: Variable Assignment - assignments: - - name: ObsErrorFactorSfcPressure/stationPressure - type: float - function: - name: ObsFunction/ObsErrorFactorSfcPressure - options: - geovar_sfc_geomz: surface_altitude - geovar_geomz: geopotential_height - station_altitude: height - - # Inflate surface pressure observation based on discrepancies between - # model and observations due to terrian - - filter: Perform Action - filter variables: - - name: stationPressure - action: - name: inflate error - inflation variable: - name: ObsErrorFactorSfcPressure/stationPressure - - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/Innovation - type: float - function: - name: ObsFunction/Arithmetic - options: - variables: - - name: ObsValue/stationPressure - - name: HofX/stationPressure - coefs: [1, -1] - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/ObsErrorBoundSfcPressure1 - type: float - function: - name: ObsFunction/ObsErrorBoundConventional - options: - obsvar: stationPressure - obserr_bound_min: 100 - obserr_bound_max: 300 - obserr_bound_factor: 5.0 - - - filter: Background Check - filter variables: - - name: stationPressure - where: - - variable: PreQC/stationPressure - is_not_in: 3 - function absolute threshold: - - name: DerivedMetaData/ObsErrorBoundSfcPressure1 - action: - name: reject - - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/ObsErrorBoundSfcPressure2 - type: float - function: - name: ObsFunction/ObsErrorBoundConventional - options: - obsvar: stationPressure - obserr_bound_min: 100 - obserr_bound_max: 300 - obserr_bound_factor: 3.5 - - - filter: Background Check - filter variables: - - name: stationPressure - where: - - variable: PreQC/stationPressure - is_in: 3 - function absolute threshold: - - name: DerivedMetaData/ObsErrorBoundSfcPressure2 - action: - name: reject - - # Inflate obs error based on duplicate check - - filter: Perform Action - filter variables: - - name: stationPressure - action: - name: inflate error - inflation variable: - name: ObsErrorFactorDuplicateCheck/stationPressure - - # Reject data based on PreUseFlag (usage in GSI) - - filter: Perform Action - filter variables: - - name: stationPressure - where: - - variable: PreUseFlag/stationPressure - is_not_in: 0, 1 - action: - name: reject - # End of Filters - - # Observation Localizations (LocalEnsembleDA) - # ------------------------------------------- - obs localizations: - - localization method: Horizontal Gaspari-Cohn - lengthscale: 1250e3 - max nobs: 10000 - - - # GeoVaLs for Driving Observation Operators (testing mode) - # -------------------------------------------------------- - geovals: - filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" - - # Passed benchmark for UFO testing - # -------------------------------- - passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml.j2 index 0ca33b1f2..f094ca94e 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml.j2 @@ -5,7 +5,7 @@ obs space: name: surface_ps distribution: - name: Halo + name: "{{distribution_type}}" halo size: 1250e3 obsdatain: engine: diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/gnssro.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/gnssro.yaml.j2 index b26d992a5..9da6bd594 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/gnssro.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/gnssro.yaml.j2 @@ -5,7 +5,7 @@ obs space: name: gnssrobndnbam distribution: - name: Halo + name: "{{distribution_type}}" halo size: 1250e3 obsdatain: engine: diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/mtiasi_metop-a.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/mtiasi_metop-a.yaml.j2 index efc4535b8..24cb44aab 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/mtiasi_metop-a.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/mtiasi_metop-a.yaml.j2 @@ -5,7 +5,7 @@ obs space: name: IASI METOP-A distribution: - name: Halo + name: "{{distribution_type}}" halo size: 1250e3 obsdatain: engine: diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/mtiasi_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/mtiasi_metop-b.yaml.j2 index 8f2f26103..7ce56ae7e 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/mtiasi_metop-b.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/mtiasi_metop-b.yaml.j2 @@ -5,7 +5,7 @@ obs space: name: IASI METOP-B distribution: - name: Halo + name: "{{distribution_type}}" halo size: 1250e3 obsdatain: engine: diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/ompsnp_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/ompsnp_npp.yaml.j2 index 488afd73b..45c49bae1 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/ompsnp_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/ompsnp_npp.yaml.j2 @@ -5,7 +5,7 @@ obs space: name: ompsnp_npp distribution: - name: Halo + name: "{{distribution_type}}" halo size: 1250e3 obsdatain: engine: diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/ompstc_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/ompstc_npp.yaml.j2 index b76b56e44..523946cda 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/ompstc_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/ompstc_npp.yaml.j2 @@ -5,7 +5,7 @@ obs space: name: ompstc_npp distribution: - name: Halo + name: "{{distribution_type}}" halo size: 1250e3 obsdatain: engine: diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-16.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-16.yaml.j2 index 1b727ad1c..e5543ba0d 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-16.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-16.yaml.j2 @@ -5,7 +5,7 @@ obs space: name: satwind_goes-16 distribution: - name: Halo + name: "{{distribution_type}}" halo size: 1250e3 obsdatain: engine: diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-18.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-18.yaml.j2 index aa394dfe7..ebbc86b02 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-18.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-18.yaml.j2 @@ -5,7 +5,7 @@ obs space: name: satwind_goes-18 distribution: - name: Halo + name: "{{distribution_type}}" halo size: 1250e3 obsdatain: engine: diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.ahi_h8.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.ahi_h8.yaml.j2 index a745ee1e3..e9ee771c3 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.ahi_h8.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.ahi_h8.yaml.j2 @@ -5,7 +5,7 @@ obs space: name: satwind_ahi_h8 distribution: - name: Halo + name: "{{distribution_type}}" halo size: 1250e3 obsdatain: engine: diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.leogeo_multi.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.leogeo_multi.yaml.j2 index c0dd03dd0..0558d66bb 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.leogeo_multi.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.leogeo_multi.yaml.j2 @@ -1,7 +1,7 @@ - obs space: name: satwind_leogeo_multi distribution: - name: Halo + name: "{{distribution_type}}" halo size: 1250e3 obsdatain: engine: diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.viirs_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.viirs_n20.yaml.j2 index b3a4abb42..2c5f1b66e 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.viirs_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.viirs_n20.yaml.j2 @@ -1,7 +1,7 @@ - obs space: name: satwind_viirs_n20 distribution: - name: Halo + name: "{{distribution_type}}" halo size: 1250e3 obsdatain: engine: diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.viirs_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.viirs_npp.yaml.j2 index 851cad1ed..18998d58d 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.viirs_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.viirs_npp.yaml.j2 @@ -1,7 +1,7 @@ - obs space: name: satwind_viirs_npp distribution: - name: Halo + name: "{{distribution_type}}" halo size: 1250e3 obsdatain: engine: diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/sondes.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/sondes.yaml.j2 index 0bc719339..dea76f050 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/sondes.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/sondes.yaml.j2 @@ -5,7 +5,7 @@ obs space: name: sondes distribution: - name: Halo + name: "{{distribution_type}}" halo size: 1250e3 obsdatain: engine: From a81988e9cf9080bd76d563d62f5e069e590c9bfe Mon Sep 17 00:00:00 2001 From: BrettHoover-NOAA <98188219+BrettHoover-NOAA@users.noreply.github.com> Date: Tue, 27 Aug 2024 09:15:11 -0500 Subject: [PATCH 048/199] Adding AHI and SEVIRI satwnd thinning, enrolling AHI-H9, SEVIRI-M9, SEVIRI-M10 (#25) This PR enrolls AHI-Himawari9, SEVIRI-METEOSAT9, and SEVIRI-METEOSAT10 satwnd types into JEDI, and establishes thinning for AHI and SEVIRI satwnd observations in their corresponding *.yaml.j2 files. This is the NOAA-EMC/jcb-gdas portion of the work, which provides the yaml.j2 files for AHI and SEVIRI satwnd types with proper thinning filter settings. This is dependent on https://github.com/NOAA-EMC/GDASApp/pull/1263, which has the corresponding bufr2ioda JSON files for enrolling the updated AHI and SEVIRI satwnd types. Thinning tests to define the filter settings can be found in https://github.com/NOAA-EMC/GDASApp/issues/1143 Co-authored-by: Brett Hoover --- .../atmosphere/satwnd.ahi_h8.yaml.j2 | 16 + .../atmosphere/satwnd.ahi_h9.yaml.j2 | 448 +++++++++++++++++ .../atmosphere/satwnd.seviri_m10.yaml.j2 | 459 +++++++++++++++++ .../atmosphere/satwnd.seviri_m11.yaml.j2 | 20 + .../atmosphere/satwnd.seviri_m8.yaml.j2 | 25 + .../atmosphere/satwnd.seviri_m9.yaml.j2 | 464 ++++++++++++++++++ 6 files changed, 1432 insertions(+) create mode 100644 parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h9.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m10.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m9.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h8.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h8.yaml.j2 index 0aac154e6..538957071 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h8.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h8.yaml.j2 @@ -222,6 +222,22 @@ action: name: reject + # THINNING: Himawari winds are not prioritized + - filter: Gaussian Thinning + horizontal_mesh: 100 + vertical_mesh: 10000 + where: + - variable: + name: MetaData/pressure + minvalue: 50001. + - filter: Gaussian Thinning + horizontal_mesh: 90 + vertical_mesh: 10000 + where: + - variable: + name: MetaData/pressure + maxvalue: 50000. + # GSI setupw routine QC # Reject any ob Type [240–260] when pressure greater than 950 mb. # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h9.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h9.yaml.j2 new file mode 100644 index 000000000..f69f13068 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h9.yaml.j2 @@ -0,0 +1,448 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: satwind_ahi_h9 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + # Observation Operator + # -------------------- + obs operator: + name: VertInterp + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: VertInterp + + # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI"s thinning of AHI/Himawari-8 satwinds + # results in more JEDI satwinds in the diag file than in GSI, but far fewer JEDI satwinds assimilated than + # GSI. JEDI under-counts assimilated winds by roughly 25-40%, relative to GSI, and this under-count is not + # even including the temporal thinning which is applied in GSI but not JEDI (by this filter below). See + # GDASApp Issue #741 for details: https://github.com/NOAA-EMC/GDASApp/issues/741 + #obs pre filters: + #- filter: Gaussian Thinning + # horizontal_mesh: 200 + # vertical_mesh: 10000 + # use_reduced_horizontal_grid: true + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: false + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Assign the initial observation error, based on height/pressure + # Hard-wiring to prepobs_errtable.global by Type + # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML + # Type 242 (Himawari VIS) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 242 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, + 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., + 7., 7., 7.] + # Type 250 (Himawari AHI WV, cloud-top or clear-sky) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 250 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, + 5., 7., 7.3, 7.6, 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., + 8., 8., 8.] + # Type Type 252 (Himawari AHI LWIR) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 252 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, + 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., + 7., 7., 7.] + # sanity-check criteria + # Observation Range Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130. + maxvalue: 130. + action: + name: reject + + # Velocity Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130. + action: + name: reject + + # GSI read routine QC (part-1) + # Exclude Type 250 with windComputationMethod==5 (clear-sky WV) --- obs tossed without passing to setup routine + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windNorthward + is_in: 250 + - variable: MetaData/windComputationMethod + is_in: 5 + action: + name: reject + + # Exclude data with satellite zenith angle > 68 for all types --- obs tossed without passing to setup routine + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: MetaData/satelliteZenithAngle + maxvalue: 68. + action: + name: reject + + # GSI read routine QC (part-2) + # Reject obs with qiWithoutForecast < 85. (also > 100., which is missing data) + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: MetaData/qiWithoutForecast + minvalue: 85. + maxvalue: 100. + action: + name: reject + + # Reject Type 252 (IR) winds with a /=0 surface type (non-water surface) when latitude > 20. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + - variable: + name: ObsType/windEastward + is_in: 252 + test variables: + - name: MetaData/latitude + maxvalue: 20. + action: + name: reject + + # THINNING: Himawari winds are not prioritized + - filter: Gaussian Thinning + horizontal_mesh: 100 + vertical_mesh: 10000 + where: + - variable: + name: MetaData/pressure + minvalue: 50001. + - filter: Gaussian Thinning + horizontal_mesh: 90 + vertical_mesh: 10000 + where: + - variable: + name: MetaData/pressure + maxvalue: 50000. + + # GSI setupw routine QC + # Reject any ob Type [240–260] when pressure greater than 950 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240-260 + test variables: + - name: MetaData/pressure + maxvalue: 95001. + action: + name: reject + + # IR (Type 242), reject when pressure is less than 700 mb + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 242 + test variables: + - name: MetaData/pressure + minvalue: 70000. + action: + name: reject + + # cloud-top WV (Type 250), reject when pressure greater than 399 mb. + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 250 + test variables: + - name: MetaData/pressure + maxvalue: 39900. + action: + name: reject + + # JMA IR (252) reject when pressure between 499 and 801 mb. + # PERFORM ACTION: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/pressure + minvalue: 49901. + maxvalue: 80099. + - variable: ObsType/windEastward + is_in: 252 + action: + name: reject + + # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation + # Turn this check off for now. + # Need to check if troposphere pressure was implemented correctly in fv3-jed + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + reference: GeoVaLs/tropopause_pressure + value: MetaData/pressure + minvalue: -5000. # 50 hPa above tropopause level, negative p-diff + action: + name: reject + + # All satwinds must adjust errors based on ObsErrorFactorPressureCheck + # prior to the SPDB check (i.e. the gross-error check). The gross-error + # check uses the adjusted errors for error-bound tightening and rejection, + # so this check has to come first. This check will inflate errors for obs + # that are too close to either the model top or bottom. + # Notes (eliu): GMAO added a required parameter: adjusted_error_name. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + surface_obs: false + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All satwinds subject to a gross-error check that contains significant + # modifiers for satwinds with a negative speed-bias. ALL wind gross-error + # checks are currently being done by the SatWindsSPDBCheck. + # CLEARED + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, + 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, + 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, + 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, + 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, + 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, + 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, + 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, + 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don"t want to inflate errors for duplication if one + # of the duplicates should be rejected. + # Notes (eliu): ObsErrorFactorDuplicateCheck obsfunction requires PreUseFlag (usage parameter from read_satwnd.f90). + # : Turn off duplicate check for now. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + # We are extending this to an additional filter that inflates final ob-errors across-the-board by + # 1/0.8 = 1.25. This is caused by the GSI value of nvqc being set to .true. in the global operational + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this nvqc functionality were to be switched off (i.e. if variational qc were to be turned off), + # you would want to remove this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + # End of Filters + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m10.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m10.yaml.j2 new file mode 100644 index 000000000..c80759fac --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m10.yaml.j2 @@ -0,0 +1,459 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: satwind_seviri_m10 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + # Observation Operator + # -------------------- + obs operator: + name: VertInterp + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: VertInterp + + # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI"s thinning of SEVIRI/METEOSAT-8 satwinds + # results in more JEDI satwinds in the diag file than in GSI, but far fewer JEDI satwinds assimilated than + # GSI. JEDI under-counts assimilated winds by roughly 25-40%, relative to GSI, and this under-count is not + # even including the temporal thinning which is applied in GSI but not JEDI (by this filter below). See + # GDASApp Issue #758 for details: https://github.com/NOAA-EMC/GDASApp/issues/758 + #obs pre filters: + #- filter: Gaussian Thinning + # where: + # - variable: ObsType/windEastward + # is_in: 243, 253 + # horizontal_mesh: 200 + # vertical_mesh: 10000 + # use_reduced_horizontal_grid: true + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: false + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Assign the initial observation error, based on height/pressure + # Hard-wiring to prepobs_errtable.global by Type + # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML + # Type 243 (MVIRI/SEVIRI VIS) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 243 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, + 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., + 7., 7., 7.] + # Type 253 (MVIRI/SEVERI LWIR) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 253 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, + 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., + 7., 7., 7.] + # Type 254 (MVIRI/SEVIRI WV, both cloud-top and clear-sky) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 254 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4.5, 6.1, + 6., 6.5, 7.3, 7.6, 7., 7.5, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., + 7., 7., 7.] + # sanity-check criteria + # Observation Range Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130. + maxvalue: 130. + action: + name: reject + + # Velocity Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130. + action: + name: reject + + # GSI read routine QC (part-1) + # Exclude Type 254 with windComputationMethod==5 (clear-sky WV) --- obs tossed without passing to setup routine + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windNorthward + is_in: 254 + - variable: MetaData/windComputationMethod + is_in: 5 + action: + name: reject + + # Exclude data with satellite zenith angle > 68 for all types --- obs tossed without passing to setup routine + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: MetaData/satelliteZenithAngle + maxvalue: 68. + action: + name: reject + + # Exclude data over non-water surface type where latitude > 20N for Type 253 (IRLW) --- obs tossed and not passed to setup routine + # Notes: This check was missing, so added (eliu) + # Replace land_type_index_NPOSS with water_area_fraction (eliu) + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 253 + - variable: MetaData/latitude + minvalue: 20. + test variables: + - name: GeoVaLs/water_area_fraction + minvalue: 0.99 + action: + name: reject + + # GSI read routine QC (part-2) + # Reject obs with qiWithoutForecast < 85. OR > 100. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: MetaData/qiWithoutForecast + minvalue: 85. + maxvalue: 100. + action: + name: reject + + # THINNING: METEOSAT winds are prioritized by qiWithoutForecast + - filter: Gaussian Thinning + horizontal_mesh: 100 + vertical_mesh: 10000 + priority_variable: + name: MetaData/qiWithoutForecast + where: + - variable: + name: MetaData/pressure + minvalue: 50001. + - filter: Gaussian Thinning + horizontal_mesh: 90 + vertical_mesh: 10000 + priority_variable: + name: MetaData/qiWithoutForecast + where: + - variable: + name: MetaData/pressure + maxvalue: 50000. + + # GSI setupw routine QC + # Reject any ob Type [240–260] when pressure greater than 950 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240-260 + test variables: + - name: MetaData/pressure + maxvalue: 95001. + action: + name: reject + + # EUMETSAT IR (253) reject when pressure between 401 and 801 mb. + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/pressure + minvalue: 40101. + maxvalue: 80099. + - variable: ObsType/windEastward + is_in: 253 + action: + name: reject + + # EUMET VIS (243) reject when pressure less than 700 mb. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 243 + test variables: + - name: MetaData/pressure + minvalue: 70000. + action: + name: reject + + # EUMET WV (254) reject when pressure greater than 399 mb. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 254 + test variables: + - name: MetaData/pressure + maxvalue: 39900. + action: + name: reject + + # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation + # Turn this check off for now. + # Need to check if troposphere pressure was implemented correctly in fv3-jed + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + reference: GeoVaLs/tropopause_pressure + value: MetaData/pressure + minvalue: -5000. # 50 hPa above tropopause level, negative p-diff + action: + name: reject + + # All satwinds must adjust errors based on ObsErrorFactorPressureCheck + # prior to the SPDB check (i.e. the gross-error check). The gross-error + # check uses the adjusted errors for error-bound tightening and rejection, + # so this check has to come first. This check will inflate errors for obs + # that are too close to either the model top or bottom. + # Notes (eliu): GMAO added a required parameter: adjusted_error_name. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + surface_obs: false + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All satwinds subject to a gross-error check that contains significant + # modifiers for satwinds with a negative speed-bias. ALL wind gross-error + # checks are currently being done by the SatWindsSPDBCheck. + # CLEARED + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, + 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, + 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, + 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, + 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, + 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, + 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, + 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, + 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don"t want to inflate errors for duplication if one + # of the duplicates should be rejected. + # Notes (eliu): ObsErrorFactorDuplicateCheck obsfunction requires PreUseFlag (usage parameter from read_satwnd.f90). + # : Turn off duplicate check for now. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + # We are extending this to an additional filter that inflates final ob-errors across-the-board by + # 1/0.8 = 1.25. This is caused by the GSI value of nvqc being set to .true. in the global operational + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this nvqc functionality were to be switched off (i.e. if variational qc were to be turned off), + # you would want to remove this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + # End of Filters + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m11.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m11.yaml.j2 index cb6a10dc0..e8bab78e8 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m11.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m11.yaml.j2 @@ -231,6 +231,26 @@ action: name: reject + # THINNING: METEOSAT winds are prioritized by qiWithoutForecast + - filter: Gaussian Thinning + horizontal_mesh: 100 + vertical_mesh: 10000 + priority_variable: + name: MetaData/qiWithoutForecast + where: + - variable: + name: MetaData/pressure + minvalue: 50001. + - filter: Gaussian Thinning + horizontal_mesh: 90 + vertical_mesh: 10000 + priority_variable: + name: MetaData/qiWithoutForecast + where: + - variable: + name: MetaData/pressure + maxvalue: 50000. + # GSI setupw routine QC # Reject any ob Type [240–260] when pressure greater than 950 mb. # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m8.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m8.yaml.j2 index e85bb815a..95d33c907 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m8.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m8.yaml.j2 @@ -231,6 +231,31 @@ action: name: reject + # THINNING: METEOSAT-8 Type-243/253 winds are thinned using priority_variable=MetaData/qiWithoutForecast + # METEOSAT-8 Type-254 winds are NOT thinned + - filter: Gaussian Thinning + horizontal_mesh: 100 + vertical_mesh: 10000 + priority_variable: + name: MetaData/qiWithoutForecast + where: + - variable: ObsType/windEastward + is_in: 243,253 + - variable: + name: MetaData/pressure + minvalue: 50001. + - filter: Gaussian Thinning + horizontal_mesh: 90 + vertical_mesh: 10000 + priority_variable: + name: MetaData/qiWithoutForecast + where: + - variable: ObsType/windEastward + is_in: 243,253 + - variable: + name: MetaData/pressure + maxvalue: 50000. + # GSI setupw routine QC # Reject any ob Type [240–260] when pressure greater than 950 mb. # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m9.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m9.yaml.j2 new file mode 100644 index 000000000..21307a34f --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m9.yaml.j2 @@ -0,0 +1,464 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: satwind_seviri_m9 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + # Observation Operator + # -------------------- + obs operator: + name: VertInterp + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: VertInterp + + # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI"s thinning of SEVIRI/METEOSAT-8 satwinds + # results in more JEDI satwinds in the diag file than in GSI, but far fewer JEDI satwinds assimilated than + # GSI. JEDI under-counts assimilated winds by roughly 25-40%, relative to GSI, and this under-count is not + # even including the temporal thinning which is applied in GSI but not JEDI (by this filter below). See + # GDASApp Issue #758 for details: https://github.com/NOAA-EMC/GDASApp/issues/758 + #obs pre filters: + #- filter: Gaussian Thinning + # where: + # - variable: ObsType/windEastward + # is_in: 243, 253 + # horizontal_mesh: 200 + # vertical_mesh: 10000 + # use_reduced_horizontal_grid: true + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: false + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Assign the initial observation error, based on height/pressure + # Hard-wiring to prepobs_errtable.global by Type + # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML + # Type 243 (MVIRI/SEVIRI VIS) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 243 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, + 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., + 7., 7., 7.] + # Type 253 (MVIRI/SEVERI LWIR) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 253 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, + 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., + 7., 7., 7.] + # Type 254 (MVIRI/SEVIRI WV, both cloud-top and clear-sky) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 254 + minvalue: -135. + maxvalue: 135. + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000., 105000., 100000., 95000., 90000., 85000., 80000., 75000., + 70000., 65000., 60000., 55000., 50000., 45000., 40000., 35000., 30000., + 25000., 20000., 15000., 10000., 7500., 5000., 4000., 3000., 2000., 1000., + 500., 400., 300., 200., 100., 0.] #Pressure (Pa) + errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4.5, 6.1, + 6., 6.5, 7.3, 7.6, 7., 7.5, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., + 7., 7., 7.] + # sanity-check criteria + # Observation Range Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130. + maxvalue: 130. + action: + name: reject + + # Velocity Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130. + action: + name: reject + + # GSI read routine QC (part-1) + # Exclude Type 254 with windComputationMethod==5 (clear-sky WV) --- obs tossed without passing to setup routine + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windNorthward + is_in: 254 + - variable: MetaData/windComputationMethod + is_in: 5 + action: + name: reject + + # Exclude data with satellite zenith angle > 68 for all types --- obs tossed without passing to setup routine + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: MetaData/satelliteZenithAngle + maxvalue: 68. + action: + name: reject + + # Exclude data over non-water surface type where latitude > 20N for Type 253 (IRLW) --- obs tossed and not passed to setup routine + # Notes: This check was missing, so added (eliu) + # Replace land_type_index_NPOSS with water_area_fraction (eliu) + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 253 + - variable: MetaData/latitude + minvalue: 20. + test variables: + - name: GeoVaLs/water_area_fraction + minvalue: 0.99 + action: + name: reject + + # GSI read routine QC (part-2) + # Reject obs with qiWithoutForecast < 85. OR > 100. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: MetaData/qiWithoutForecast + minvalue: 85. + maxvalue: 100. + action: + name: reject + + # THINNING: METEOSAT-9 Type-243/253 winds are thinned using priority_variable=MetaData/qiWithoutForecast + # METEOSAT-9 Type-254 winds are NOT thinned + - filter: Gaussian Thinning + horizontal_mesh: 100 + vertical_mesh: 10000 + priority_variable: + name: MetaData/qiWithoutForecast + where: + - variable: ObsType/windEastward + is_in: 243,253 + - variable: + name: MetaData/pressure + minvalue: 50001. + - filter: Gaussian Thinning + horizontal_mesh: 90 + vertical_mesh: 10000 + priority_variable: + name: MetaData/qiWithoutForecast + where: + - variable: ObsType/windEastward + is_in: 243,253 + - variable: + name: MetaData/pressure + maxvalue: 50000. + + # GSI setupw routine QC + # Reject any ob Type [240–260] when pressure greater than 950 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240-260 + test variables: + - name: MetaData/pressure + maxvalue: 95001. + action: + name: reject + + # EUMETSAT IR (253) reject when pressure between 401 and 801 mb. + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: MetaData/pressure + minvalue: 40101. + maxvalue: 80099. + - variable: ObsType/windEastward + is_in: 253 + action: + name: reject + + # EUMET VIS (243) reject when pressure less than 700 mb. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 243 + test variables: + - name: MetaData/pressure + minvalue: 70000. + action: + name: reject + + # EUMET WV (254) reject when pressure greater than 399 mb. + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 254 + test variables: + - name: MetaData/pressure + maxvalue: 39900. + action: + name: reject + + # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation + # Turn this check off for now. + # Need to check if troposphere pressure was implemented correctly in fv3-jed + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + reference: GeoVaLs/tropopause_pressure + value: MetaData/pressure + minvalue: -5000. # 50 hPa above tropopause level, negative p-diff + action: + name: reject + + # All satwinds must adjust errors based on ObsErrorFactorPressureCheck + # prior to the SPDB check (i.e. the gross-error check). The gross-error + # check uses the adjusted errors for error-bound tightening and rejection, + # so this check has to come first. This check will inflate errors for obs + # that are too close to either the model top or bottom. + # Notes (eliu): GMAO added a required parameter: adjusted_error_name. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + surface_obs: false + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All satwinds subject to a gross-error check that contains significant + # modifiers for satwinds with a negative speed-bias. ALL wind gross-error + # checks are currently being done by the SatWindsSPDBCheck. + # CLEARED + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, + 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, + 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, + 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, + 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, + 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, + 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, + 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, + 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don"t want to inflate errors for duplication if one + # of the duplicates should be rejected. + # Notes (eliu): ObsErrorFactorDuplicateCheck obsfunction requires PreUseFlag (usage parameter from read_satwnd.f90). + # : Turn off duplicate check for now. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + # We are extending this to an additional filter that inflates final ob-errors across-the-board by + # 1/0.8 = 1.25. This is caused by the GSI value of nvqc being set to .true. in the global operational + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this nvqc functionality were to be switched off (i.e. if variational qc were to be turned off), + # you would want to remove this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + # End of Filters + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 From 50fed660a09be746dc4eb1848e9bec76fc91f4ce Mon Sep 17 00:00:00 2001 From: Guillaume Vernieres Date: Tue, 17 Sep 2024 08:51:07 -0400 Subject: [PATCH 049/199] Revive the marine jcb implementation (#26) Some updates/fixes to the marine yaml templates. --- .../marine/soca_2cice_antarctic.yaml.j2 | 43 +++++++++++++++++++ .../marine/soca_2cice_arctic.yaml.j2 | 40 +++++++++++++++++ ...caincr2mom6.yaml => socaincr2mom6.yaml.j2} | 10 ++--- ..._background_error_static_diffusion.yaml.j2 | 12 +++--- 4 files changed, 94 insertions(+), 11 deletions(-) create mode 100644 parm/jcb-gdas/algorithm/marine/soca_2cice_antarctic.yaml.j2 create mode 100644 parm/jcb-gdas/algorithm/marine/soca_2cice_arctic.yaml.j2 rename parm/jcb-gdas/algorithm/marine/{socaincr2mom6.yaml => socaincr2mom6.yaml.j2} (64%) diff --git a/parm/jcb-gdas/algorithm/marine/soca_2cice_antarctic.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_2cice_antarctic.yaml.j2 new file mode 100644 index 000000000..73fa36753 --- /dev/null +++ b/parm/jcb-gdas/algorithm/marine/soca_2cice_antarctic.yaml.j2 @@ -0,0 +1,43 @@ +input geometry: + mom6_input_nml: mom_input.nml + fields metadata: fields_metadata.yaml + geom_grid_file: soca_gridspec.nc + +output geometry: + mom6_input_nml: mom_input.nml + fields metadata: fields_metadata.yaml + geom_grid_file: soca_gridspec.nc + +variable change: + variable change name: Soca2Cice + do inverse: false + seaice edge: 0.9 + shuffle: false # seg. fault when true + rescale prior: + min hice: 0.5 + min hsno: 0.1 + domain: antarctic + cice background state: + restart: {{ ice_rst }} + grid: ' ' + ncat: 5 + ice_lev: 7 + sno_lev: 1 + tstep: PT1H + cice output: + restart: {{ ice_rst }} + output variables: [tocn, socn, hocn, cicen, hicen, hsnon] + +states: +- input: + read_from_file: 1 + basename: ./ + ocn_filename: {{ ocn_ana }} + ice_filename: {{ ice_ana }} + date: '{{ fcst_begin }}' + state variables: [tocn, socn, hocn, cicen, hicen, hsnon] + output: + datadir: Data + exp: soca2cice + type: fc + date: '{{ fcst_begin }}' diff --git a/parm/jcb-gdas/algorithm/marine/soca_2cice_arctic.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_2cice_arctic.yaml.j2 new file mode 100644 index 000000000..e686eea08 --- /dev/null +++ b/parm/jcb-gdas/algorithm/marine/soca_2cice_arctic.yaml.j2 @@ -0,0 +1,40 @@ +input geometry: + mom6_input_nml: mom_input.nml + fields metadata: ./fields_metadata.yaml + +output geometry: + mom6_input_nml: mom_input.nml + fields metadata: ./fields_metadata.yaml + +variable change: + variable change name: Soca2Cice + do inverse: false + seaice edge: 0.8 + shuffle: false + rescale prior: + min hice: 0.5 + min hsno: 0.1 + domain: arctic + cice background state: + restart: {{ ice_rst }} + ncat: 5 + ice_lev: 7 + sno_lev: 1 + tstep: PT1H + cice output: + restart: {{ ice_rst }} + output variables: [tocn, socn, hocn, cicen, hicen, hsnon] + +states: +- input: + read_from_file: 1 + basename: ./ + ocn_filename: {{ ocn_ana }} + ice_filename: {{ ice_ana }} + date: '{{ fcst_begin }}' + state variables: [tocn, socn, hocn, cicen, hicen, hsnon] + output: + datadir: Data + exp: soca2cice + type: fc + date: '{{ fcst_begin }}' diff --git a/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml b/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 similarity index 64% rename from parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml rename to parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 index 83ce7048e..ececb6bea 100644 --- a/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml +++ b/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 @@ -2,27 +2,27 @@ geometry: mom6_input_nml: mom_input.nml fields metadata: ./fields_metadata.yaml -date: '{{marine_window_begin}}' +date: '{{ marine_window_begin }}' layers variable: [hocn] increment variables: [tocn, socn, uocn, vocn, ssh] vertical geometry: - date: '{{marine_window_begin}}' + date: '{{ marine_window_begin }}' basename: ./INPUT/ ocn_filename: MOM.res.nc read_from_file: 1 soca increment: - date: '{{marine_window_begin}}' + date: '{{ marine_window_begin }}' basename: ./Data/ - ocn_filename: 'ocn.3dvarfgat_pseudo.incr.{{ATM_WINDOW_MIDDLE}}.nc' + ocn_filename: 'ocn.3dvarfgat_pseudo.incr.{{marine_window_middle}}.nc' read_from_file: 1 output increment: datadir: ./ - date: '{{marine_window_begin}}' + date: '{{ marine_window_begin }}' exp: mom6_iau type: incr output file: inc.nc diff --git a/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 b/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 index a2d7e0547..4275f3e91 100644 --- a/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 +++ b/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 @@ -18,21 +18,21 @@ saber central block: groups: - name: ocean horizontal: - filename: hz_ocean.nc + filename: ./staticb/hz_ocean.nc vertical: - filename: vt_ocean.nc + filename: ./staticb/vt_ocean.nc - name: ice horizontal: - filename: hz_ice.nc + filename: ./staticb/hz_ice.nc saber outer blocks: - saber block name: StdDev read: model file: date: '{{marine_stddev_time}}' - basename: ./ - ocn_filename: 'ocn.bkgerr_stddev.incr.{{marine_stddev_time}}.nc' - ice_filename: 'ice.bkgerr_stddev.incr.{{marine_stddev_time}}.nc' + basename: ./staticb/ + ocn_filename: 'ocn.bkgerr_stddev.nc' + ice_filename: 'ice.bkgerr_stddev.nc' read_from_file: 3 linear variable change: From fca3e6784cc36ffcdc2b4c19092f7d5d57c18906 Mon Sep 17 00:00:00 2001 From: DavidNew-NOAA <134300700+DavidNew-NOAA@users.noreply.github.com> Date: Wed, 25 Sep 2024 05:07:05 -0400 Subject: [PATCH 050/199] Change hydrostatic_delz to delz (#27) --- .../jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 | 2 +- .../algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 index f69bca233..4e6406088 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 @@ -1,7 +1,7 @@ variable change: variable change name: Model2GeoVaLs input variables: &bkgvars [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr,surface_geopotential_height] - output variables: &fv3incrvars [ua,va,t,sphum,ice_wat,liq_wat,o3mr,delp,hydrostatic_delz] + output variables: &fv3incrvars [ua,va,t,sphum,ice_wat,liq_wat,o3mr,delp,delz] jedi increment variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] fv3 increment variables: *fv3incrvars background geometry: diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 index d5277f129..80a36349a 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 @@ -1,7 +1,7 @@ variable change: variable change name: Model2GeoVaLs input variables: &bkgvars [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr,surface_geopotential_height] - output variables: &fv3incrvars [ua,va,t,sphum,ice_wat,liq_wat,o3mr,delp,hydrostatic_delz] + output variables: &fv3incrvars [ua,va,t,sphum,ice_wat,liq_wat,o3mr,delp,delz] jedi increment variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] fv3 increment variables: *fv3incrvars background geometry: From 5a7445508ce9fd1f957f4bc80ff5f803b718a433 Mon Sep 17 00:00:00 2001 From: Guillaume Vernieres Date: Wed, 2 Oct 2024 20:48:03 -0400 Subject: [PATCH 051/199] Updates to the marine static-b yaml (#30) Adjusting yaml to reflect the switch to using `oops::utils::FieldSetHelpers::write/readFieldSet` ... I fail to see the point of this repo, what was wrong with keeping the yamls in the gdasapp? --- ..._background_error_static_diffusion.yaml.j2 | 26 +++++-------------- 1 file changed, 7 insertions(+), 19 deletions(-) diff --git a/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 b/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 index 4275f3e91..bb5de6194 100644 --- a/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 +++ b/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 @@ -1,29 +1,17 @@ covariance model: SABER saber central block: - saber block name: EXPLICIT_DIFFUSION - active variables: [tocn, socn, ssh, cicen] - geometry: - mom6_input_nml: mom_input.nml - fields metadata: ./fields_metadata.yaml - group mapping: - - name: ocean - variables: - - tocn - - socn - - ssh - - name: ice - variables: - - cicen + saber block name: diffusion read: groups: - - name: ocean + - variables: [tocn, socn, ssh] horizontal: - filename: ./staticb/hz_ocean.nc + filepath: ./staticb/hz_ocean vertical: - filename: ./staticb/vt_ocean.nc - - name: ice + levels: {{marine_vt_levels}} + filepath: ./staticb/vt_ocean + - variables: [cicen] horizontal: - filename: ./staticb/hz_ice.nc + filepath: ./staticb/hz_ice saber outer blocks: - saber block name: StdDev From 9b89b8ca73dd2e7838f6a8a2fa96a483c1335b98 Mon Sep 17 00:00:00 2001 From: DavidNew-NOAA <134300700+DavidNew-NOAA@users.noreply.github.com> Date: Wed, 9 Oct 2024 15:30:45 -0400 Subject: [PATCH 052/199] JCB-based obs+bias staging (#31) This PR is a companion to Global Workflow PR [#2992](https://github.com/NOAA-EMC/global-workflow/pull/2992) and GDASApp PR [#1312](https://github.com/NOAA-EMC/GDASApp/pull/1312). It add observation and bias files staging templates required for the above Global Workflow PR. --- .../algorithm/atmosphere/atm_bias_staging.yaml.j2 | 10 ++++++++++ .../algorithm/atmosphere/atm_obs_staging.yaml.j2 | 8 ++++++++ 2 files changed, 18 insertions(+) create mode 100644 parm/jcb-gdas/algorithm/atmosphere/atm_bias_staging.yaml.j2 create mode 100644 parm/jcb-gdas/algorithm/atmosphere/atm_obs_staging.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/atmosphere/atm_bias_staging.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/atm_bias_staging.yaml.j2 new file mode 100644 index 000000000..c91ea2e80 --- /dev/null +++ b/parm/jcb-gdas/algorithm/atmosphere/atm_bias_staging.yaml.j2 @@ -0,0 +1,10 @@ +mkdir: + - '{{atmosphere_obsbiasout_path}}' +copy: + {% for observation_from_jcb in observations %} + {% if use_observer(observation_from_jcb) %} + {% if observation_from_jcb in bias_files %} + - ['{{atmosphere_obsbiasroot_path}}/{{atmosphere_obsbiasin_prefix}}{{bias_files[observation_from_jcb]}}', '{{atmosphere_obsbiasin_path}}'] + {% endif %} + {% endif %} + {% endfor %} diff --git a/parm/jcb-gdas/algorithm/atmosphere/atm_obs_staging.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/atm_obs_staging.yaml.j2 new file mode 100644 index 000000000..34d098490 --- /dev/null +++ b/parm/jcb-gdas/algorithm/atmosphere/atm_obs_staging.yaml.j2 @@ -0,0 +1,8 @@ +mkdir: + - '{{atmosphere_obsdatain_path}}' +copy: + {% for observation_from_jcb in observations %} + {% if use_observer(observation_from_jcb) %} + - ['{{atmosphere_obsdataroot_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}', '{{atmosphere_obsdatain_path}}'] + {% endif %} + {% endfor %} From feb900d9905f5c1f371bf513a5f6bcbbeae7c1d8 Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Tue, 22 Oct 2024 10:17:28 -0400 Subject: [PATCH 053/199] Add support for global aerosol DA in JCB (#33) Add in templates so that global aerosol DA can use JCB to generate YAML configuration for the variational application + B matrix supporting executables. --- .../aero/aero_convert_background.yaml.j2 | 48 +++++++++++ .../aero/aero_gen_bmatrix_diagb.yaml.j2 | 81 +++++++++++++++++++ .../aero/aero_gen_bmatrix_diffusion.yaml.j2 | 48 +++++++++++ .../aero/aero_3dfgat_outer_loop_1.yaml.j2 | 1 + .../aero/aero_3dfgat_outer_loop_2.yaml.j2 | 1 + .../aero/aero_3dvar_outer_loop_1.yaml.j2 | 17 ++++ .../aero/aero_3dvar_outer_loop_2.yaml.j2 | 17 ++++ .../model/aero/aero_background.yaml.j2 | 10 +++ ..._background_error_static_diffusion.yaml.j2 | 33 ++++++++ .../aero_final_increment_cubed_sphere.yaml.j2 | 23 ++++++ .../aero_final_increment_gaussian.yaml.j2 | 18 +++++ .../jcb-gdas/model/aero/aero_geometry.yaml.j2 | 0 .../aero/aero_geometry_background.yaml.j2 | 11 +++ .../model/aero/aero_model_pseudo.yaml.j2 | 13 +++ .../observations/aero/viirs_n20_aod.yaml.j2 | 63 +++++++++++++++ .../observations/aero/viirs_n21_aod.yaml.j2 | 63 +++++++++++++++ .../observations/aero/viirs_npp_aod.yaml.j2 | 63 +++++++++++++++ 17 files changed, 510 insertions(+) create mode 100644 parm/jcb-gdas/algorithm/aero/aero_convert_background.yaml.j2 create mode 100644 parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diagb.yaml.j2 create mode 100644 parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diffusion.yaml.j2 create mode 120000 parm/jcb-gdas/model/aero/aero_3dfgat_outer_loop_1.yaml.j2 create mode 120000 parm/jcb-gdas/model/aero/aero_3dfgat_outer_loop_2.yaml.j2 create mode 100644 parm/jcb-gdas/model/aero/aero_3dvar_outer_loop_1.yaml.j2 create mode 100644 parm/jcb-gdas/model/aero/aero_3dvar_outer_loop_2.yaml.j2 create mode 100644 parm/jcb-gdas/model/aero/aero_background.yaml.j2 create mode 100644 parm/jcb-gdas/model/aero/aero_background_error_static_diffusion.yaml.j2 create mode 100644 parm/jcb-gdas/model/aero/aero_final_increment_cubed_sphere.yaml.j2 create mode 100644 parm/jcb-gdas/model/aero/aero_final_increment_gaussian.yaml.j2 delete mode 100644 parm/jcb-gdas/model/aero/aero_geometry.yaml.j2 create mode 100644 parm/jcb-gdas/model/aero/aero_geometry_background.yaml.j2 create mode 100644 parm/jcb-gdas/model/aero/aero_model_pseudo.yaml.j2 create mode 100644 parm/jcb-gdas/observations/aero/viirs_n20_aod.yaml.j2 create mode 100644 parm/jcb-gdas/observations/aero/viirs_n21_aod.yaml.j2 create mode 100644 parm/jcb-gdas/observations/aero/viirs_npp_aod.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/aero/aero_convert_background.yaml.j2 b/parm/jcb-gdas/algorithm/aero/aero_convert_background.yaml.j2 new file mode 100644 index 000000000..0a1344fb7 --- /dev/null +++ b/parm/jcb-gdas/algorithm/aero/aero_convert_background.yaml.j2 @@ -0,0 +1,48 @@ +input geometry: + fms initialization: + namelist filename: "{{aero_fv3jedi_files_path}}/fmsmpp.nml" + field table filename: "{{aero_fv3jedi_files_path}}/field_table" + akbk: "{{aero_fv3jedi_files_path}}/akbk.nc4" + layout: + - {{ aero_layout_x }} + - {{ aero_layout_y }} + npx: {{ aero_npx_ges }} + npy: {{ aero_npy_ges }} + npz: {{ aero_npz_ges }} + field metadata override: "{{aero_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" +output geometry: + fms initialization: + namelist filename: "{{aero_fv3jedi_files_path}}/fmsmpp.nml" + field table filename: "{{aero_fv3jedi_files_path}}/field_table" + akbk: "{{aero_fv3jedi_files_path}}/akbk.nc4" + layout: + - {{ aero_layout_x }} + - {{ aero_layout_y }} + npx: {{ aero_npx_anl }} + npy: {{ aero_npy_anl }} + npz: {{ aero_npz_anl }} + field metadata override: "{{aero_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" +states: +- input: + datetime: '{{ aero_background_error_time_iso }}' + filetype: fms restart + state variables: [mass_fraction_of_sulfate_in_air, + mass_fraction_of_hydrophobic_black_carbon_in_air, + mass_fraction_of_hydrophilic_black_carbon_in_air, + mass_fraction_of_hydrophobic_organic_carbon_in_air, + mass_fraction_of_hydrophilic_organic_carbon_in_air, + mass_fraction_of_dust001_in_air, mass_fraction_of_dust002_in_air, + mass_fraction_of_dust003_in_air, mass_fraction_of_dust004_in_air, + mass_fraction_of_dust005_in_air, mass_fraction_of_sea_salt001_in_air, + mass_fraction_of_sea_salt002_in_air, mass_fraction_of_sea_salt003_in_air, + mass_fraction_of_sea_salt004_in_air] + datapath: {{ aero_background_path }} + filename_core: '{{ aero_background_error_time_fv3 }}.fv_core.res.nc' + filename_trcr: '{{ aero_background_error_time_fv3 }}.fv_tracer.res.nc' + filename_cplr: '{{ aero_background_error_time_fv3 }}.coupler.res' + output: + filetype: fms restart + datapath: {{ aero_background_path }} + filename_core: 'anlres.fv_core.res.nc' + filename_trcr: 'anlres.fv_tracer.res.nc' + filename_cplr: 'anlres.coupler.res' diff --git a/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diagb.yaml.j2 b/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diagb.yaml.j2 new file mode 100644 index 000000000..d1f3edac2 --- /dev/null +++ b/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diagb.yaml.j2 @@ -0,0 +1,81 @@ +geometry: + fms initialization: + namelist filename: "{{aero_fv3jedi_files_path}}/fmsmpp.nml" + field table filename: "{{aero_fv3jedi_files_path}}/field_table" + akbk: "{{aero_fv3jedi_files_path}}/akbk.nc4" + layout: + - {{ aero_layout_x }} + - {{ aero_layout_y }} + npx: {{ aero_npx_anl }} + npy: {{ aero_npy_anl }} + npz: {{ aero_npz_anl }} + field metadata override: "{{aero_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" +date: '{{ aero_background_error_time_iso }}' +background: + datetime: '{{ aero_background_error_time_iso }}' + filetype: fms restart + state variables: [mass_fraction_of_sulfate_in_air, + mass_fraction_of_hydrophobic_black_carbon_in_air, + mass_fraction_of_hydrophilic_black_carbon_in_air, + mass_fraction_of_hydrophobic_organic_carbon_in_air, + mass_fraction_of_hydrophilic_organic_carbon_in_air, + mass_fraction_of_dust001_in_air, mass_fraction_of_dust002_in_air, + mass_fraction_of_dust003_in_air, mass_fraction_of_dust004_in_air, + mass_fraction_of_dust005_in_air, mass_fraction_of_sea_salt001_in_air, + mass_fraction_of_sea_salt002_in_air, mass_fraction_of_sea_salt003_in_air, + mass_fraction_of_sea_salt004_in_air] + datapath: {{ aero_background_path }} + filename_core: '{{ aero_background_error_time_fv3 }}.anlres.fv_core.res.nc' + filename_trcr: '{{ aero_background_error_time_fv3 }}.anlres.fv_tracer.res.nc' + filename_cplr: '{{ aero_background_error_time_fv3 }}.anlres.coupler.res' +background error: + filetype: fms restart + datapath: {{ aero_standard_deviation_path }} + filename_trcr: stddev.fv_tracer.res.nc + filename_cplr: stddev.coupler.res + +climate background error: + geometry: + fms initialization: + namelist filename: "{{aero_fv3jedi_files_path}}/fmsmpp.nml" + field table filename: "{{aero_fv3jedi_files_path}}/field_table" + akbk: "{{aero_fv3jedi_files_path}}/akbk.nc4" + layout: + - {{ aero_layout_x }} + - {{ aero_layout_y }} + npx: {{ aero_npx_clim_b }} + npy: {{ aero_npy_clim_b }} + npz: {{ aero_npz_clim_b }} + field metadata override: "{{aero_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" + climate background error stddev: + filetype: fms restart + skip coupler file: true + datapath: {{ aero_climatological_b_path }} + filename_trcr: stddev.fv_tracer.res.nc + filename_cplr: stddev.coupler.res + diagb weight: {{ aero_diagb_weight }} + staticb rescaling factor: {{ aero_diagb_static_rescale_factor }} + +variables: + name: + - mass_fraction_of_sulfate_in_air + - mass_fraction_of_hydrophobic_black_carbon_in_air + - mass_fraction_of_hydrophilic_black_carbon_in_air + - mass_fraction_of_hydrophobic_organic_carbon_in_air + - mass_fraction_of_hydrophilic_organic_carbon_in_air + - mass_fraction_of_dust001_in_air + - mass_fraction_of_dust002_in_air + - mass_fraction_of_dust003_in_air + - mass_fraction_of_dust004_in_air + - mass_fraction_of_dust005_in_air + - mass_fraction_of_sea_salt001_in_air + - mass_fraction_of_sea_salt002_in_air + - mass_fraction_of_sea_salt003_in_air + - mass_fraction_of_sea_salt004_in_air + +rescale: {{ aero_diagb_rescale_factor }} # rescales the filtered std. dev. by "rescale" +number of halo points: {{ aero_diagb_n_halo }} +number of neighbors: {{ aero_diagb_n_neighbors }} +simple smoothing: + horizontal iterations: {{ aero_diagb_smooth_horiz_iter }} + vertical iterations: {{ aero_diagb_smooth_vert_iter }} diff --git a/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diffusion.yaml.j2 b/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diffusion.yaml.j2 new file mode 100644 index 000000000..0462daaca --- /dev/null +++ b/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diffusion.yaml.j2 @@ -0,0 +1,48 @@ +geometry: + fms initialization: + namelist filename: "{{aero_fv3jedi_files_path}}/fmsmpp.nml" + field table filename: "{{aero_fv3jedi_files_path}}/field_table" + akbk: "{{aero_fv3jedi_files_path}}/akbk.nc4" + layout: + - {{ aero_layout_x }} + - {{ aero_layout_y }} + npx: {{ aero_npx_anl }} + npy: {{ aero_npy_anl }} + npz: {{ aero_npz_anl }} + field metadata override: "{{aero_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" +date: '{{ aero_background_error_time_iso }}' +background: + datetime: '{{ aero_background_error_time_iso }}' + filetype: fms restart + state variables: [mass_fraction_of_sulfate_in_air, + mass_fraction_of_hydrophobic_black_carbon_in_air, + mass_fraction_of_hydrophilic_black_carbon_in_air, + mass_fraction_of_hydrophobic_organic_carbon_in_air, + mass_fraction_of_hydrophilic_organic_carbon_in_air, + mass_fraction_of_dust001_in_air, mass_fraction_of_dust002_in_air, + mass_fraction_of_dust003_in_air, mass_fraction_of_dust004_in_air, + mass_fraction_of_dust005_in_air, mass_fraction_of_sea_salt001_in_air, + mass_fraction_of_sea_salt002_in_air, mass_fraction_of_sea_salt003_in_air, + mass_fraction_of_sea_salt004_in_air] + datapath: {{ aero_background_path }} + filename_core: '{{ aero_background_error_time_fv3 }}.anlres.fv_core.res.nc' + filename_trcr: '{{ aero_background_error_time_fv3 }}.anlres.fv_tracer.res.nc' + filename_cplr: '{{ aero_background_error_time_fv3 }}.anlres.coupler.res' +background error: + covariance model: SABER + saber central block: + saber block name: diffusion + calibration: + normalization: + iterations: {{ aero_diffusion_iter }} + groups: + - horizontal: + fixed value: {{ aero_diffusion_horiz_len }} + write: + filepath: "{{ aero_berror_diffusion_directory }}/diffusion_hz" + - vertical: + levels: {{ aero_npz_ges }} + fixed value: {{ aero_diffusion_fixed_val }} + as gaussian: true + write: + filepath: "{{ aero_berror_diffusion_directory }}/diffusion_vt" diff --git a/parm/jcb-gdas/model/aero/aero_3dfgat_outer_loop_1.yaml.j2 b/parm/jcb-gdas/model/aero/aero_3dfgat_outer_loop_1.yaml.j2 new file mode 120000 index 000000000..be2ee77a8 --- /dev/null +++ b/parm/jcb-gdas/model/aero/aero_3dfgat_outer_loop_1.yaml.j2 @@ -0,0 +1 @@ +aero_3dvar_outer_loop_1.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/model/aero/aero_3dfgat_outer_loop_2.yaml.j2 b/parm/jcb-gdas/model/aero/aero_3dfgat_outer_loop_2.yaml.j2 new file mode 120000 index 000000000..d029b98df --- /dev/null +++ b/parm/jcb-gdas/model/aero/aero_3dfgat_outer_loop_2.yaml.j2 @@ -0,0 +1 @@ +aero_3dvar_outer_loop_2.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/model/aero/aero_3dvar_outer_loop_1.yaml.j2 b/parm/jcb-gdas/model/aero/aero_3dvar_outer_loop_1.yaml.j2 new file mode 100644 index 000000000..68529a1be --- /dev/null +++ b/parm/jcb-gdas/model/aero/aero_3dvar_outer_loop_1.yaml.j2 @@ -0,0 +1,17 @@ +- ninner: 35 + gradient norm reduction: 1e-10 + test: on + geometry: + fms initialization: + namelist filename: {{aero_fv3jedi_files_path}}/fmsmpp.nml + field table filename: {{aero_fv3jedi_files_path}}/field_table + akbk: {{aero_fv3jedi_files_path}}/akbk.nc4 + layout: + - {{aero_layout_x}} + - {{aero_layout_y}} + npx: {{aero_npx_anl}} + npy: {{aero_npy_anl}} + npz: {{aero_npz_anl}} + field metadata override: {{aero_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml + diagnostics: + departures: bkgmob diff --git a/parm/jcb-gdas/model/aero/aero_3dvar_outer_loop_2.yaml.j2 b/parm/jcb-gdas/model/aero/aero_3dvar_outer_loop_2.yaml.j2 new file mode 100644 index 000000000..7ca1a03f1 --- /dev/null +++ b/parm/jcb-gdas/model/aero/aero_3dvar_outer_loop_2.yaml.j2 @@ -0,0 +1,17 @@ +- ninner: 35 + gradient norm reduction: 1e-10 + test: on + geometry: + fms initialization: + namelist filename: {{aero_fv3jedi_files_path}}/fmsmpp.nml + field table filename: {{aero_fv3jedi_files_path}}/field_table + akbk: {{aero_fv3jedi_files_path}}/akbk.nc4 + layout: + - {{aero_layout_x}} + - {{aero_layout_y}} + npx: {{aero_npx_anl}} + npy: {{aero_npy_anl}} + npz: {{aero_npz_anl}} + field metadata override: {{aero_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml + diagnostics: + departures: bkgmob1 diff --git a/parm/jcb-gdas/model/aero/aero_background.yaml.j2 b/parm/jcb-gdas/model/aero/aero_background.yaml.j2 new file mode 100644 index 000000000..012c50db5 --- /dev/null +++ b/parm/jcb-gdas/model/aero/aero_background.yaml.j2 @@ -0,0 +1,10 @@ +datapath: {{ aero_background_path }} +filetype: fms restart +datetime: '{{ aero_background_time_iso }}' +filename is datetime templated: true +filename_core: '%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc' +filename_trcr: '%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc' +filename_cplr: '%yyyy%mm%dd.%hh%MM%ss.coupler.res' +state variables: [t,delp,sphum,so4,bc1,bc2,oc1,oc2, + dust1,dust2,dust3,dust4,dust5, + seas1,seas2,seas3,seas4] diff --git a/parm/jcb-gdas/model/aero/aero_background_error_static_diffusion.yaml.j2 b/parm/jcb-gdas/model/aero/aero_background_error_static_diffusion.yaml.j2 new file mode 100644 index 000000000..73af01bb8 --- /dev/null +++ b/parm/jcb-gdas/model/aero/aero_background_error_static_diffusion.yaml.j2 @@ -0,0 +1,33 @@ +covariance model: SABER +saber central block: + saber block name: diffusion + read: + groups: + - variables: [mass_fraction_of_sulfate_in_air, + mass_fraction_of_hydrophobic_black_carbon_in_air, + mass_fraction_of_hydrophilic_black_carbon_in_air, + mass_fraction_of_hydrophobic_organic_carbon_in_air, + mass_fraction_of_hydrophilic_organic_carbon_in_air, + mass_fraction_of_dust001_in_air, mass_fraction_of_dust002_in_air, + mass_fraction_of_dust003_in_air, mass_fraction_of_dust004_in_air, + mass_fraction_of_dust005_in_air, mass_fraction_of_sea_salt001_in_air, + mass_fraction_of_sea_salt002_in_air, mass_fraction_of_sea_salt003_in_air, + mass_fraction_of_sea_salt004_in_air] + horizontal: + filepath: "{{aero_berror_data_directory}}/diffusion_hz" + vertical: + levels: {{aero_npz_anl}} + filepath: "{{aero_berror_data_directory}}/diffusion_vt" +saber outer blocks: +- saber block name: StdDev + read: + model file: + datetime: "{{ aero_cycle_time_iso }}" + set datetime on read: true + filetype: fms restart + psinfile: true + datapath: "{{aero_berror_data_directory}}" + filename_core: '{{ aero_cycle_time_fv3 }}.stddev.fv_core.res.nc' + filename_trcr: '{{ aero_cycle_time_fv3 }}.stddev.fv_tracer.res.nc' + filename_cplr: '{{ aero_cycle_time_fv3 }}.stddev.coupler.res' + date: "{{ aero_cycle_time_iso }}" diff --git a/parm/jcb-gdas/model/aero/aero_final_increment_cubed_sphere.yaml.j2 b/parm/jcb-gdas/model/aero/aero_final_increment_cubed_sphere.yaml.j2 new file mode 100644 index 000000000..e34a7fcfa --- /dev/null +++ b/parm/jcb-gdas/model/aero/aero_final_increment_cubed_sphere.yaml.j2 @@ -0,0 +1,23 @@ +geometry: + fms initialization: + namelist filename: "{{aero_fv3jedi_files_path}}/fmsmpp.nml" + field table filename: "{{aero_fv3jedi_files_path}}/field_table" + akbk: "{{aero_fv3jedi_files_path}}/akbk.nc4" + layout: + - {{aero_layout_x}} + - {{aero_layout_y}} + npx: {{aero_npx_anl}} + npy: {{aero_npy_anl}} + npz: {{aero_npz_anl}} + field metadata override: "{{aero_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" +output: + state component: + datapath: {{ aero_analysis_path }} + prefix: aeroinc + filetype: fms restart + filename_core: '{{ aero_cycle_time_fv3 }}.fv_core.res.nc' + filename_trcr: '{{ aero_cycle_time_fv3 }}.fv_tracer.res.nc' + filename_cplr: '{{ aero_cycle_time_fv3 }}.coupler.res' + state variables: [t,delp,sphum,so4,bc1,bc2,oc1,oc2, + dust1,dust2,dust3,dust4,dust5, + seas1,seas2,seas3,seas4] diff --git a/parm/jcb-gdas/model/aero/aero_final_increment_gaussian.yaml.j2 b/parm/jcb-gdas/model/aero/aero_final_increment_gaussian.yaml.j2 new file mode 100644 index 000000000..97ffc8d4a --- /dev/null +++ b/parm/jcb-gdas/model/aero/aero_final_increment_gaussian.yaml.j2 @@ -0,0 +1,18 @@ +grid type: regular gaussian +local interpolator type: atlas interpolator +interpolation method: + type: finite-element +number of latitude gridpoints: {{ aero_npy_ges - 1 }} +variables to output: [mass_fraction_of_sulfate_in_air, + mass_fraction_of_hydrophobic_black_carbon_in_air, + mass_fraction_of_hydrophilic_black_carbon_in_air, + mass_fraction_of_hydrophobic_organic_carbon_in_air, + mass_fraction_of_hydrophilic_organic_carbon_in_air, + mass_fraction_of_dust001_in_air, mass_fraction_of_dust002_in_air, + mass_fraction_of_dust003_in_air, mass_fraction_of_dust004_in_air, + mass_fraction_of_dust005_in_air, mass_fraction_of_sea_salt001_in_air, + mass_fraction_of_sea_salt002_in_air, mass_fraction_of_sea_salt003_in_air, + mass_fraction_of_sea_salt004_in_air] +all model levels: true +datapath: {{ aero_analysis_path }} +prefix: aeroinc_gauss diff --git a/parm/jcb-gdas/model/aero/aero_geometry.yaml.j2 b/parm/jcb-gdas/model/aero/aero_geometry.yaml.j2 deleted file mode 100644 index e69de29bb..000000000 diff --git a/parm/jcb-gdas/model/aero/aero_geometry_background.yaml.j2 b/parm/jcb-gdas/model/aero/aero_geometry_background.yaml.j2 new file mode 100644 index 000000000..834e2fcca --- /dev/null +++ b/parm/jcb-gdas/model/aero/aero_geometry_background.yaml.j2 @@ -0,0 +1,11 @@ +fms initialization: + namelist filename: "{{aero_fv3jedi_files_path}}/fmsmpp.nml" + field table filename: "{{aero_fv3jedi_files_path}}/field_table" +akbk: "{{aero_fv3jedi_files_path}}/akbk.nc4" +layout: +- {{aero_layout_x}} +- {{aero_layout_y}} +npx: {{aero_npx_ges}} +npy: {{aero_npy_ges}} +npz: {{aero_npz_ges}} +field metadata override: "{{aero_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" diff --git a/parm/jcb-gdas/model/aero/aero_model_pseudo.yaml.j2 b/parm/jcb-gdas/model/aero/aero_model_pseudo.yaml.j2 new file mode 100644 index 000000000..61af8db2a --- /dev/null +++ b/parm/jcb-gdas/model/aero/aero_model_pseudo.yaml.j2 @@ -0,0 +1,13 @@ +name: PSEUDO +datapath: {{ aero_background_path}} +filetype: fms restart +filename is datetime templated: true +filename_core: '%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc' +filename_trcr: '%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc' +filename_sfcd: '%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc' +filename_sfcw: '%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc' +filename_cplr: '%yyyy%mm%dd.%hh%MM%ss.coupler.res' +tstep: {{aero_forecast_timestep}} +model variables: [t,delp,sphum,so4,bc1,bc2,oc1,oc2, + dust1,dust2,dust3,dust4,dust5, + seas1,seas2,seas3,seas4] diff --git a/parm/jcb-gdas/observations/aero/viirs_n20_aod.yaml.j2 b/parm/jcb-gdas/observations/aero/viirs_n20_aod.yaml.j2 new file mode 100644 index 000000000..5ef6aca35 --- /dev/null +++ b/parm/jcb-gdas/observations/aero/viirs_n20_aod.yaml.j2 @@ -0,0 +1,63 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: viirs_n20_aod + obsdatain: + engine: + type: H5File + obsfile: "{{aero_obsdatain_path}}/{{aero_obsdatain_prefix}}{{observation_from_jcb}}{{aero_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{aero_obsdataout_path}}/{{aero_obsdataout_prefix}}{{observation_from_jcb}}{{aero_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [aerosolOpticalDepth] + channels: 4 + + # Observation Operator + # -------------------- + get values: + interpolation method: barycentric + time interpolation: linear + obs operator: + name: AodCRTM + Absorbers: [H2O,O3] + obs options: + Sensor_ID: &{{observation_from_jcb}}_sensor_id v.viirs-m_j1 + EndianType: little_endian + AerosolOption: aerosols_gocart_default + CoefficientPath: "{{crtm_coefficient_path}}" + obs error: + covariance model: diagonal + + # Observation Bias Correction (VarBC) + # ----------------------------------- + + # Observation Filters (QC) + # ------------------------ + obs filters: + - filter: PreQC + maxvalue: 1 + - filter: Domain Check + where: + - variable: + name: latitude@MetaData + minvalue: -60 + maxvalue: 60 + - filter: Bounds Check + filter variables: + - name: aerosolOpticalDepth + channels: 4 + minvalue: 0 + maxvalue: 4.9 + action: + name: reject + - filter: Background Check + channels: 4 + threshold: 3.0 + action: + name: inflate error + inflation factor: 3.0 diff --git a/parm/jcb-gdas/observations/aero/viirs_n21_aod.yaml.j2 b/parm/jcb-gdas/observations/aero/viirs_n21_aod.yaml.j2 new file mode 100644 index 000000000..9ea41402b --- /dev/null +++ b/parm/jcb-gdas/observations/aero/viirs_n21_aod.yaml.j2 @@ -0,0 +1,63 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: viirs_n21_aod + obsdatain: + engine: + type: H5File + obsfile: "{{aero_obsdatain_path}}/{{aero_obsdatain_prefix}}{{observation_from_jcb}}{{aero_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{aero_obsdataout_path}}/{{aero_obsdataout_prefix}}{{observation_from_jcb}}{{aero_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [aerosolOpticalDepth] + channels: 4 + + # Observation Operator + # -------------------- + get values: + interpolation method: barycentric + time interpolation: linear + obs operator: + name: AodCRTM + Absorbers: [H2O,O3] + obs options: + Sensor_ID: &{{observation_from_jcb}}_sensor_id v.viirs-m_j2 + EndianType: little_endian + AerosolOption: aerosols_gocart_default + CoefficientPath: "{{crtm_coefficient_path}}" + obs error: + covariance model: diagonal + + # Observation Bias Correction (VarBC) + # ----------------------------------- + + # Observation Filters (QC) + # ------------------------ + obs filters: + - filter: PreQC + maxvalue: 1 + - filter: Domain Check + where: + - variable: + name: latitude@MetaData + minvalue: -60 + maxvalue: 60 + - filter: Bounds Check + filter variables: + - name: aerosolOpticalDepth + channels: 4 + minvalue: 0 + maxvalue: 4.9 + action: + name: reject + - filter: Background Check + channels: 4 + threshold: 3.0 + action: + name: inflate error + inflation factor: 3.0 diff --git a/parm/jcb-gdas/observations/aero/viirs_npp_aod.yaml.j2 b/parm/jcb-gdas/observations/aero/viirs_npp_aod.yaml.j2 new file mode 100644 index 000000000..57cad00a1 --- /dev/null +++ b/parm/jcb-gdas/observations/aero/viirs_npp_aod.yaml.j2 @@ -0,0 +1,63 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: viirs_npp_aod + obsdatain: + engine: + type: H5File + obsfile: "{{aero_obsdatain_path}}/{{aero_obsdatain_prefix}}{{observation_from_jcb}}{{aero_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{aero_obsdataout_path}}/{{aero_obsdataout_prefix}}{{observation_from_jcb}}{{aero_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [aerosolOpticalDepth] + channels: 4 + + # Observation Operator + # -------------------- + get values: + interpolation method: barycentric + time interpolation: linear + obs operator: + name: AodCRTM + Absorbers: [H2O,O3] + obs options: + Sensor_ID: &{{observation_from_jcb}}_sensor_id v.viirs-m_npp + EndianType: little_endian + AerosolOption: aerosols_gocart_default + CoefficientPath: "{{crtm_coefficient_path}}" + obs error: + covariance model: diagonal + + # Observation Bias Correction (VarBC) + # ----------------------------------- + + # Observation Filters (QC) + # ------------------------ + obs filters: + - filter: PreQC + maxvalue: 1 + - filter: Domain Check + where: + - variable: + name: latitude@MetaData + minvalue: -60 + maxvalue: 60 + - filter: Bounds Check + filter variables: + - name: aerosolOpticalDepth + channels: 4 + minvalue: 0 + maxvalue: 4.9 + action: + name: reject + - filter: Background Check + channels: 4 + threshold: 3.0 + action: + name: inflate error + inflation factor: 3.0 From 139a549c3532b8022807e54435c7f7b50adc084f Mon Sep 17 00:00:00 2001 From: Dan Holdaway <27729500+danholdaway@users.noreply.github.com> Date: Wed, 23 Oct 2024 11:53:27 -0400 Subject: [PATCH 054/199] Add safety feature for testing algo changes (#35) When the JCB tests are being run (from anywhere) the code will clone the main JCB repo and when doing so it will look for the branch name of the PR. Then it will run a script to clone all the clients that are registered for testing as well as the algorithms repo. This script will use the branch name of the main JCB repo and if found for any clients clone that branch. Now imagine this scenario: someone is making changes to their client (say gdas) and simultaneously making changes to jcb-algorithms. They might realize that they need to change another client to accommodate the algorithm changes so they make a branch with the same name in (say) rrfs. However if they do not have that branch in the main jcb repo there wouldn't exist a mechanism to check out the correct branch of jcb-algorithms and the other clients during the integration tests. We could put that kind of logic in the GitHub script but that would be complicated and have to propagate to every jcb repo. The change in this code forces the user to make a branch of the main jcb repo if they have created the same branch in the jcb-algorithms branch and thus provide all the mechanisms for the scripts to check everything out properly. The algorithms should really be part of JCB since they are generic and it would avoid the need to do this. But because of EE2 compliance we can't have YAMLs with source code, which is why they are in a separate repo. Co-authored-by: danholdaway --- .../workflows/run_jcb_basic_testing.yaml | 28 +++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml b/parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml index 97714eba5..4457e675f 100644 --- a/parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml +++ b/parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml @@ -52,6 +52,34 @@ jobs: echo "JCB_BRANCH=develop" >> $GITHUB_ENV fi + - name: Check for the branch name in the jcb-algorithms repo + run: | + BRANCH_NAME=${{ env.JCB_APP_BRANCH }} + if git ls-remote --heads https://github.com/NOAA-EMC/jcb-algorithms.git $BRANCH_NAME | grep -q "refs/heads/$BRANCH_NAME"; then + echo "Branch $BRANCH_NAME exists in jcb-algorithms repo." + echo "JCB_ALGO_BRANCH=$BRANCH_NAME" >> $GITHUB_ENV + + # If the branch exists in jcb-algorithms repo but JCB_BRANCH is develop then we need to + # throw an error. This is not a safe situation. The developer should create a branch in + # the main jcb repo with the same name as the branch in the jcb-application repo. + # If there were branches in other apps to account for the changes in the algorithm repo + # they could not be tested here since the scipt is not clever enough to check for + # the existence of the branch being tested here in all the clients. The safest thing to do + # is simply create a branch with the same name (even if empty) in the main jcb repo. This + # will ensure the branches of the other applications are checked out by the init script. + + if [ "${{ env.JCB_BRANCH }}" == "develop" ]; then + echo "Branch $BRANCH_NAME exists in jcb-algorithms repo but not in the main jcb repo. " + echo "Please create a branch with the same name (even if empty with no PR) in the main " + echo "jcb repo. This ensures safely checking all the clients that depend on the " + echo "jcb and jcb-algorithms repos with the changes being proposed." + exit 1 + fi + else + echo "Branch $BRANCH_NAME does not exist in jcb repo. Using develop branch." + echo "JCB_ALGO_BRANCH=develop" >> $GITHUB_ENV + fi + - name: Clone jcb repository run: | mkdir -p empty_hooks From 409f6a9b0fa14b8e483b858db60002913ad39788 Mon Sep 17 00:00:00 2001 From: danholdaway Date: Wed, 23 Oct 2024 12:00:22 -0400 Subject: [PATCH 055/199] Logic fails post merge --- .../.github/workflows/run_jcb_basic_testing.yaml | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml b/parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml index 4457e675f..102146ca5 100644 --- a/parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml +++ b/parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml @@ -52,9 +52,17 @@ jobs: echo "JCB_BRANCH=develop" >> $GITHUB_ENV fi - - name: Check for the branch name in the jcb-algorithms repo + - name: Check for the branch name in the jcb-algorithms repo (if not develop) run: | BRANCH_NAME=${{ env.JCB_APP_BRANCH }} + + # If branch name is develop then we don't need to check the jcb-algorithms repo. + if [ "$BRANCH_NAME" == "develop" ]; then + echo "JCB_ALGO_BRANCH=develop" >> $GITHUB_ENV + exit 0 + fi + + # If the branch is not develop then check the jcb-algorithms repo. if git ls-remote --heads https://github.com/NOAA-EMC/jcb-algorithms.git $BRANCH_NAME | grep -q "refs/heads/$BRANCH_NAME"; then echo "Branch $BRANCH_NAME exists in jcb-algorithms repo." echo "JCB_ALGO_BRANCH=$BRANCH_NAME" >> $GITHUB_ENV @@ -75,9 +83,6 @@ jobs: echo "jcb and jcb-algorithms repos with the changes being proposed." exit 1 fi - else - echo "Branch $BRANCH_NAME does not exist in jcb repo. Using develop branch." - echo "JCB_ALGO_BRANCH=develop" >> $GITHUB_ENV fi - name: Clone jcb repository From d262cc244f69d17312f57fb64221565c580f8d5d Mon Sep 17 00:00:00 2001 From: emilyhcliu <36091766+emilyhcliu@users.noreply.github.com> Date: Wed, 23 Oct 2024 12:04:39 -0400 Subject: [PATCH 056/199] Remove obselete entries and add "reduce obs space" option iasi filter (#29) This PR proposes the following changes for IASI: 1. Remove obsolete entries related to QC flags in the observation filter 2. Rename mtiasi to iasi 3. Add `reduce obs space There is a companion [GDASApp PR #1292](https://github.com/NOAA-EMC/GDASApp/pull/1292) This partly resolves Issue https://github.com/NOAA-EMC/GDASApp/issues/1291 Co-authored-by: Emily Liu Co-authored-by: Dan Holdaway <27729500+danholdaway@users.noreply.github.com> --- ...{mtiasi_metop-a.yaml => iasi_metop-a.yaml} | 0 ...{mtiasi_metop-b.yaml => iasi_metop-b.yaml} | 0 ...i_metop-a.yaml.j2 => iasi_metop-a.yaml.j2} | 120 +++++------------ ...i_metop-b.yaml.j2 => iasi_metop-b.yaml.j2} | 120 +++++------------ ...i_metop-a.yaml.j2 => iasi_metop-a.yaml.j2} | 121 +++++------------- ...i_metop-b.yaml.j2 => iasi_metop-b.yaml.j2} | 121 +++++------------- 6 files changed, 124 insertions(+), 358 deletions(-) rename parm/jcb-gdas/observation_chronicle/atmosphere/{mtiasi_metop-a.yaml => iasi_metop-a.yaml} (100%) rename parm/jcb-gdas/observation_chronicle/atmosphere/{mtiasi_metop-b.yaml => iasi_metop-b.yaml} (100%) rename parm/jcb-gdas/observations/atmosphere-lgetkf/{mtiasi_metop-a.yaml.j2 => iasi_metop-a.yaml.j2} (92%) rename parm/jcb-gdas/observations/atmosphere-lgetkf/{mtiasi_metop-b.yaml.j2 => iasi_metop-b.yaml.j2} (92%) rename parm/jcb-gdas/observations/atmosphere/{mtiasi_metop-a.yaml.j2 => iasi_metop-a.yaml.j2} (92%) rename parm/jcb-gdas/observations/atmosphere/{mtiasi_metop-b.yaml.j2 => iasi_metop-b.yaml.j2} (92%) diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/mtiasi_metop-a.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/iasi_metop-a.yaml similarity index 100% rename from parm/jcb-gdas/observation_chronicle/atmosphere/mtiasi_metop-a.yaml rename to parm/jcb-gdas/observation_chronicle/atmosphere/iasi_metop-a.yaml diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/mtiasi_metop-b.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/iasi_metop-b.yaml similarity index 100% rename from parm/jcb-gdas/observation_chronicle/atmosphere/mtiasi_metop-b.yaml rename to parm/jcb-gdas/observation_chronicle/atmosphere/iasi_metop-b.yaml diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/mtiasi_metop-a.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/iasi_metop-a.yaml.j2 similarity index 92% rename from parm/jcb-gdas/observations/atmosphere-lgetkf/mtiasi_metop-a.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere-lgetkf/iasi_metop-a.yaml.j2 index 24cb44aab..9edba023a 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/mtiasi_metop-a.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/iasi_metop-a.yaml.j2 @@ -70,40 +70,7 @@ # Observation Pre Filters (QC) # ---------------------------- obs pre filters: - # Step 0-A: Create Diagnostic Flags - #- filter: Create Diagnostic Flags - # filter variables: - # - name: brightnessTemperature - # channels: *{{observation_from_jcb}}_simulated_channels - # flags: - # - name: ScanEdgeRemoval - # initial value: false - # force reinitialization: false - # - name: Thinning - # initial value: false - # force reinitialization: false - # - name: ShortwaveIRCheck - # initial value: false - # force reinitialization: false - # - name: ObsValueRangeCheck - # initial value: false - # force reinitialization: false - # - name: CloudDetection - # initial value: false - # force reinitialization: false - # - name: NSSTCheck - # initial value: false - # force reinitialization: false - # - name: GrossCheck - # initial value: false - # force reinitialization: false - # - name: InterChannelConsistency - # initial value: false - # force reinitialization: false - # - name: UseFlagCheck - # initial value: false - # force reinitialization: false - # Step 0-B: Create Derived Variables + # Step 1: Create Derived Variables # Assign channel wavenumbers in m-1 - filter: Variable Assignment assignments: @@ -120,7 +87,7 @@ intercept: 64475.0 use channel numbers: true - # Transform radiance to brightness temperature + # Step 2: Transform radiance to brightness temperature - filter: Variable Transforms Transform: SatBrightnessTempFromRad transform from: @@ -133,11 +100,7 @@ planck1: 1.191042953e-16 planck2: 1.4387774e-2 - # Step 0-C: Assign Observation Error - - # Observation Prior Filters (QC) - # ------------------------------ - obs prior filters: + # Step 3: Assign Observation Error - filter: Perform Action filter variables: - name: brightnessTemperature @@ -193,10 +156,7 @@ 2.5, 2.5, 2.52, 2.52, 2.54, 2.5, 2.48, 2.5, 2.55, 2.5, 2.48, 2.5, 2.5, 2.52, 2.52, 2.48, 2.5, 2.5, 2.52, 2.46, 2.53, 9] - # Observation Post Filters (QC) - # ----------------------------- - obs post filters: - # Step 1: Remove Observations from the Edge of the Scan + # Step 4: Remove Observations from the Edge of the Scan - filter: Domain Check filter variables: - name: brightnessTemperature @@ -207,11 +167,9 @@ is_in: 5-56 action: name: reject - # actions: - # - name: set - # flag: ScanEdgeRemoval - # - name: reject - # Step 2: Data Thinning + name: reduce obs space + + # Step 5: Data Thinning - filter: Gaussian Thinning filter variables: - name: brightnessTemperature @@ -224,11 +182,12 @@ # partition_longitude_bins_using_mesh: true action: name: reject - # actions: - # - name: set - # flag: Thinning - # - name: reject - # Step 3: Wavenumber Check + name: reduce obs space + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Step 6: Wavenumber Check - filter: Perform Action filter variables: - name: brightnessTemperature @@ -246,11 +205,8 @@ minvalue: 1.0e-12 action: name: reject - # actions: - # - name: set - # flag: ShortwaveIRCheck - # - name: reject - # Step 4: Observation Error Inflation based on Wavenumber + + # Step 7: Observation Error Inflation based on Wavenumber - filter: Perform Action filter variables: - name: brightnessTemperature @@ -262,21 +218,19 @@ channels: *{{observation_from_jcb}}_simulated_channels options: channels: *{{observation_from_jcb}}_simulated_channels - # Step 5: Observation Range Sanity Check + + # Step 8: Observation Range Sanity Check + # Valid range: (50, 550) - filter: Bounds Check filter variables: - name: brightnessTemperature channels: *{{observation_from_jcb}}_simulated_channels minvalue: 50.00001 - maxvalue: 449.99999 - # maxvalue: 100.00000 + maxvalue: 549.99999 action: name: reject - # actions: - # - name: set - # flag: ObsValueRangeCheck - # - name: reject - # Step 6: Topography Check + + # Step 9: Topography Check - filter: Perform Action filter variables: - name: brightnessTemperature @@ -290,7 +244,7 @@ channels: *{{observation_from_jcb}}_simulated_channels sensor: *{{observation_from_jcb}}_sensor_id - # Step 7: Transmittance Top Check + # Step 10: Transmittance Top Check - filter: Perform Action filter variables: - name: brightnessTemperature @@ -302,7 +256,8 @@ channels: *{{observation_from_jcb}}_simulated_channels options: channels: *{{observation_from_jcb}}_simulated_channels - # Step 8: Cloud Detection Check + + # Step 11: Cloud Detection Check - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -382,11 +337,8 @@ maxvalue: 1.0e-12 action: name: reject - # actions: - # - name: set - # flag: CloudDetection - # - name: reject - # Step 9: NSST Retrieval Check + + # Step 12: NSST Retrieval Check - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -433,11 +385,8 @@ maxvalue: 1.0e-12 action: name: reject - # actions: - # - name: set - # flag: NSSTCheck - # - name: reject - # Step 10: Surface Jacobians Check + + # Step 13: Surface Jacobians Check - filter: Perform Action filter variables: - name: brightnessTemperature @@ -453,7 +402,7 @@ obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] sensor: *{{observation_from_jcb}}_sensor_id - # Step 11: Gross check + # Step 14: Gross check - filter: Background Check filter variables: - name: brightnessTemperature @@ -517,11 +466,8 @@ error parameter vector: *{{observation_from_jcb}}_oberr action: name: reject - # actions: - # - name: set - # flag: GrossCheck - # - name: reject - # Step 12: Useflag Check + + # Step 15: Useflag Check - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -566,10 +512,6 @@ minvalue: 1.0e-12 action: name: reject - # actions: - # - name: set - # flag: UseFlagCheck - # - name: reject # Observation Localizations (LocalEnsembleDA) # ------------------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/mtiasi_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/iasi_metop-b.yaml.j2 similarity index 92% rename from parm/jcb-gdas/observations/atmosphere-lgetkf/mtiasi_metop-b.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere-lgetkf/iasi_metop-b.yaml.j2 index 7ce56ae7e..d22bdbcfa 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/mtiasi_metop-b.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/iasi_metop-b.yaml.j2 @@ -70,40 +70,7 @@ # Observation Pre Filters (QC) # ---------------------------- obs pre filters: - # Step 0-A: Create Diagnostic Flags - #- filter: Create Diagnostic Flags - # filter variables: - # - name: brightnessTemperature - # channels: *{{observation_from_jcb}}_simulated_channels - # flags: - # - name: ScanEdgeRemoval - # initial value: false - # force reinitialization: false - # - name: Thinning - # initial value: false - # force reinitialization: false - # - name: ShortwaveIRCheck - # initial value: false - # force reinitialization: false - # - name: ObsValueRangeCheck - # initial value: false - # force reinitialization: false - # - name: CloudDetection - # initial value: false - # force reinitialization: false - # - name: NSSTCheck - # initial value: false - # force reinitialization: false - # - name: GrossCheck - # initial value: false - # force reinitialization: false - # - name: InterChannelConsistency - # initial value: false - # force reinitialization: false - # - name: UseFlagCheck - # initial value: false - # force reinitialization: false - # Step 0-B: Create Derived Variables + # Step 1: Create Derived Variables # Assign channel wavenumbers in m-1 - filter: Variable Assignment assignments: @@ -120,7 +87,7 @@ intercept: 64475.0 use channel numbers: true - # Transform radiance to brightness temperature + # Step 2: Transform radiance to brightness temperature - filter: Variable Transforms Transform: SatBrightnessTempFromRad transform from: @@ -133,11 +100,7 @@ planck1: 1.191042953e-16 planck2: 1.4387774e-2 - # Step 0-C: Assign Observation Error - - # Observation Prior Filters (QC) - # ------------------------------ - obs prior filters: + # Step 3: Assign Observation Error - filter: Perform Action filter variables: - name: brightnessTemperature @@ -193,10 +156,7 @@ 2.5, 2.5, 2.52, 2.52, 2.54, 2.5, 2.48, 2.5, 2.55, 2.5, 2.48, 2.5, 2.5, 2.52, 2.52, 2.48, 2.5, 2.5, 2.52, 2.46, 2.53, 9] - # Observation Post Filters (QC) - # ----------------------------- - obs post filters: - # Step 1: Remove Observations from the Edge of the Scan + # Step 4: Remove Observations from the Edge of the Scan - filter: Domain Check filter variables: - name: brightnessTemperature @@ -207,11 +167,9 @@ is_in: 5-56 action: name: reject - # actions: - # - name: set - # flag: ScanEdgeRemoval - # - name: reject - # Step 2: Data Thinning + name: reduce obs space + + # Step 5: Data Thinning - filter: Gaussian Thinning filter variables: - name: brightnessTemperature @@ -224,11 +182,12 @@ # partition_longitude_bins_using_mesh: true action: name: reject - # actions: - # - name: set - # flag: Thinning - # - name: reject - # Step 3: Wavenumber Check + name: reduce obs space + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Step 6: Wavenumber Check - filter: Perform Action filter variables: - name: brightnessTemperature @@ -246,11 +205,8 @@ minvalue: 1.0e-12 action: name: reject - # actions: - # - name: set - # flag: ShortwaveIRCheck - # - name: reject - # Step 4: Observation Error Inflation based on Wavenumber + + # Step 7: Observation Error Inflation based on Wavenumber - filter: Perform Action filter variables: - name: brightnessTemperature @@ -262,21 +218,19 @@ channels: *{{observation_from_jcb}}_simulated_channels options: channels: *{{observation_from_jcb}}_simulated_channels - # Step 5: Observation Range Sanity Check + + # Step 8: Observation Range Sanity Check + # Valid range: (50, 550) - filter: Bounds Check filter variables: - name: brightnessTemperature channels: *{{observation_from_jcb}}_simulated_channels minvalue: 50.00001 - maxvalue: 449.99999 - # maxvalue: 100.00000 + maxvalue: 549.99999 action: name: reject - # actions: - # - name: set - # flag: ObsValueRangeCheck - # - name: reject - # Step 6: Topography Check + + # Step 9: Topography Check - filter: Perform Action filter variables: - name: brightnessTemperature @@ -290,7 +244,7 @@ channels: *{{observation_from_jcb}}_simulated_channels sensor: *{{observation_from_jcb}}_sensor_id - # Step 7: Transmittance Top Check + # Step 10: Transmittance Top Check - filter: Perform Action filter variables: - name: brightnessTemperature @@ -302,7 +256,8 @@ channels: *{{observation_from_jcb}}_simulated_channels options: channels: *{{observation_from_jcb}}_simulated_channels - # Step 8: Cloud Detection Check + + # Step 11: Cloud Detection Check - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -382,11 +337,8 @@ maxvalue: 1.0e-12 action: name: reject - # actions: - # - name: set - # flag: CloudDetection - # - name: reject - # Step 9: NSST Retrieval Check + + # Step 12: NSST Retrieval Check - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -433,11 +385,8 @@ maxvalue: 1.0e-12 action: name: reject - # actions: - # - name: set - # flag: NSSTCheck - # - name: reject - # Step 10: Surface Jacobians Check + + # Step 13: Surface Jacobians Check - filter: Perform Action filter variables: - name: brightnessTemperature @@ -453,7 +402,7 @@ obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] sensor: *{{observation_from_jcb}}_sensor_id - # Step 11: Gross check + # Step 14: Gross check - filter: Background Check filter variables: - name: brightnessTemperature @@ -517,11 +466,8 @@ error parameter vector: *{{observation_from_jcb}}_oberr action: name: reject - # actions: - # - name: set - # flag: GrossCheck - # - name: reject - # Step 12: Useflag Check + + # Step 15: Useflag Check - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -566,10 +512,6 @@ minvalue: 1.0e-12 action: name: reject - # actions: - # - name: set - # flag: UseFlagCheck - # - name: reject # Observation Localizations (LocalEnsembleDA) # ------------------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/mtiasi_metop-a.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 similarity index 92% rename from parm/jcb-gdas/observations/atmosphere/mtiasi_metop-a.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 index 8e1ae98b4..890e2705d 100644 --- a/parm/jcb-gdas/observations/atmosphere/mtiasi_metop-a.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 @@ -67,40 +67,7 @@ # Observation Pre Filters (QC) # ---------------------------- obs pre filters: - # Step 0-A: Create Diagnostic Flags - #- filter: Create Diagnostic Flags - # filter variables: - # - name: brightnessTemperature - # channels: *{{observation_from_jcb}}_simulated_channels - # flags: - # - name: ScanEdgeRemoval - # initial value: false - # force reinitialization: false - # - name: Thinning - # initial value: false - # force reinitialization: false - # - name: ShortwaveIRCheck - # initial value: false - # force reinitialization: false - # - name: ObsValueRangeCheck - # initial value: false - # force reinitialization: false - # - name: CloudDetection - # initial value: false - # force reinitialization: false - # - name: NSSTCheck - # initial value: false - # force reinitialization: false - # - name: GrossCheck - # initial value: false - # force reinitialization: false - # - name: InterChannelConsistency - # initial value: false - # force reinitialization: false - # - name: UseFlagCheck - # initial value: false - # force reinitialization: false - # Step 0-B: Create Derived Variables + # Step 1: Create Derived Variables # Assign channel wavenumbers in m-1 - filter: Variable Assignment assignments: @@ -117,7 +84,7 @@ intercept: 64475.0 use channel numbers: true - # Transform radiance to brightness temperature + # Step 2: Transform radiance to brightness temperature - filter: Variable Transforms Transform: SatBrightnessTempFromRad transform from: @@ -130,11 +97,7 @@ planck1: 1.191042953e-16 planck2: 1.4387774e-2 - # Step 0-C: Assign Observation Error - - # Observation Prior Filters (QC) - # ------------------------------ - obs prior filters: + # Step 3: Assign Observation Error - filter: Perform Action filter variables: - name: brightnessTemperature @@ -190,10 +153,7 @@ 2.5, 2.5, 2.52, 2.52, 2.54, 2.5, 2.48, 2.5, 2.55, 2.5, 2.48, 2.5, 2.5, 2.52, 2.52, 2.48, 2.5, 2.5, 2.52, 2.46, 2.53, 9] - # Observation Post Filters (QC) - # ----------------------------- - obs post filters: - # Step 1: Remove Observations from the Edge of the Scan + # Step 4: Remove Observations from the Edge of the Scan - filter: Domain Check filter variables: - name: brightnessTemperature @@ -204,11 +164,9 @@ is_in: 5-56 action: name: reject - # actions: - # - name: set - # flag: ScanEdgeRemoval - # - name: reject - # Step 2: Data Thinning + name: reduce obs space + + # Step 5: Data Thinning - filter: Gaussian Thinning filter variables: - name: brightnessTemperature @@ -221,11 +179,12 @@ # partition_longitude_bins_using_mesh: true action: name: reject - # actions: - # - name: set - # flag: Thinning - # - name: reject - # Step 3: Wavenumber Check + name: reduce obs space + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Step 6: Wavenumber Check - filter: Perform Action filter variables: - name: brightnessTemperature @@ -243,11 +202,8 @@ minvalue: 1.0e-12 action: name: reject - # actions: - # - name: set - # flag: ShortwaveIRCheck - # - name: reject - # Step 4: Observation Error Inflation based on Wavenumber + + # Step 7: Observation Error Inflation based on Wavenumber - filter: Perform Action filter variables: - name: brightnessTemperature @@ -259,21 +215,19 @@ channels: *{{observation_from_jcb}}_simulated_channels options: channels: *{{observation_from_jcb}}_simulated_channels - # Step 5: Observation Range Sanity Check + + # Step 8: Observation Range Sanity Check + # Valid range: (50, 550) - filter: Bounds Check filter variables: - name: brightnessTemperature channels: *{{observation_from_jcb}}_simulated_channels minvalue: 50.00001 - maxvalue: 449.99999 - # maxvalue: 100.00000 + maxvalue: 549.99999 action: name: reject - # actions: - # - name: set - # flag: ObsValueRangeCheck - # - name: reject - # Step 6: Topography Check + + # Step 9: Topography Check - filter: Perform Action filter variables: - name: brightnessTemperature @@ -287,7 +241,7 @@ channels: *{{observation_from_jcb}}_simulated_channels sensor: *{{observation_from_jcb}}_sensor_id - # Step 7: Transmittance Top Check + # Step 10: Transmittance Top Check - filter: Perform Action filter variables: - name: brightnessTemperature @@ -299,7 +253,8 @@ channels: *{{observation_from_jcb}}_simulated_channels options: channels: *{{observation_from_jcb}}_simulated_channels - # Step 8: Cloud Detection Check + + # Step 11: Cloud Detection Check - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -379,11 +334,8 @@ maxvalue: 1.0e-12 action: name: reject - # actions: - # - name: set - # flag: CloudDetection - # - name: reject - # Step 9: NSST Retrieval Check + + # Step 12: NSST Retrieval Check - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -430,11 +382,8 @@ maxvalue: 1.0e-12 action: name: reject - # actions: - # - name: set - # flag: NSSTCheck - # - name: reject - # Step 10: Surface Jacobians Check + + # Step 13: Surface Jacobians Check - filter: Perform Action filter variables: - name: brightnessTemperature @@ -450,7 +399,7 @@ obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] sensor: *{{observation_from_jcb}}_sensor_id - # Step 11: Gross check + # Step 14: Gross check - filter: Background Check filter variables: - name: brightnessTemperature @@ -514,11 +463,8 @@ error parameter vector: *{{observation_from_jcb}}_oberr action: name: reject - # actions: - # - name: set - # flag: GrossCheck - # - name: reject - # Step 12: Useflag Check + + # Step 15: Useflag Check - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -563,11 +509,6 @@ minvalue: 1.0e-12 action: name: reject - # actions: - # - name: set - # flag: UseFlagCheck - # - name: reject - # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/mtiasi_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 similarity index 92% rename from parm/jcb-gdas/observations/atmosphere/mtiasi_metop-b.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 index dd1cff322..57e9236b3 100644 --- a/parm/jcb-gdas/observations/atmosphere/mtiasi_metop-b.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 @@ -67,40 +67,7 @@ # Observation Pre Filters (QC) # ---------------------------- obs pre filters: - # Step 0-A: Create Diagnostic Flags - #- filter: Create Diagnostic Flags - # filter variables: - # - name: brightnessTemperature - # channels: *{{observation_from_jcb}}_simulated_channels - # flags: - # - name: ScanEdgeRemoval - # initial value: false - # force reinitialization: false - # - name: Thinning - # initial value: false - # force reinitialization: false - # - name: ShortwaveIRCheck - # initial value: false - # force reinitialization: false - # - name: ObsValueRangeCheck - # initial value: false - # force reinitialization: false - # - name: CloudDetection - # initial value: false - # force reinitialization: false - # - name: NSSTCheck - # initial value: false - # force reinitialization: false - # - name: GrossCheck - # initial value: false - # force reinitialization: false - # - name: InterChannelConsistency - # initial value: false - # force reinitialization: false - # - name: UseFlagCheck - # initial value: false - # force reinitialization: false - # Step 0-B: Create Derived Variables + # Step 1: Create Derived Variables # Assign channel wavenumbers in m-1 - filter: Variable Assignment assignments: @@ -117,7 +84,7 @@ intercept: 64475.0 use channel numbers: true - # Transform radiance to brightness temperature + # Step 2: Transform radiance to brightness temperature - filter: Variable Transforms Transform: SatBrightnessTempFromRad transform from: @@ -130,11 +97,7 @@ planck1: 1.191042953e-16 planck2: 1.4387774e-2 - # Step 0-C: Assign Observation Error - - # Observation Prior Filters (QC) - # ------------------------------ - obs prior filters: + # Step 3: Assign Observation Error - filter: Perform Action filter variables: - name: brightnessTemperature @@ -190,10 +153,7 @@ 2.5, 2.5, 2.52, 2.52, 2.54, 2.5, 2.48, 2.5, 2.55, 2.5, 2.48, 2.5, 2.5, 2.52, 2.52, 2.48, 2.5, 2.5, 2.52, 2.46, 2.53, 9] - # Observation Post Filters (QC) - # ----------------------------- - obs post filters: - # Step 1: Remove Observations from the Edge of the Scan + # Step 4: Remove Observations from the Edge of the Scan - filter: Domain Check filter variables: - name: brightnessTemperature @@ -204,11 +164,9 @@ is_in: 5-56 action: name: reject - # actions: - # - name: set - # flag: ScanEdgeRemoval - # - name: reject - # Step 2: Data Thinning + name: reduce obs space + + # Step 5: Data Thinning - filter: Gaussian Thinning filter variables: - name: brightnessTemperature @@ -221,11 +179,12 @@ # partition_longitude_bins_using_mesh: true action: name: reject - # actions: - # - name: set - # flag: Thinning - # - name: reject - # Step 3: Wavenumber Check + name: reduce obs space + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Step 6: Wavenumber Check - filter: Perform Action filter variables: - name: brightnessTemperature @@ -243,11 +202,8 @@ minvalue: 1.0e-12 action: name: reject - # actions: - # - name: set - # flag: ShortwaveIRCheck - # - name: reject - # Step 4: Observation Error Inflation based on Wavenumber + + # Step 7: Observation Error Inflation based on Wavenumber - filter: Perform Action filter variables: - name: brightnessTemperature @@ -259,21 +215,19 @@ channels: *{{observation_from_jcb}}_simulated_channels options: channels: *{{observation_from_jcb}}_simulated_channels - # Step 5: Observation Range Sanity Check + + # Step 8: Observation Range Sanity Check + # Valid range: (50, 550) - filter: Bounds Check filter variables: - name: brightnessTemperature channels: *{{observation_from_jcb}}_simulated_channels minvalue: 50.00001 - maxvalue: 449.99999 - # maxvalue: 100.00000 + maxvalue: 549.99999 action: name: reject - # actions: - # - name: set - # flag: ObsValueRangeCheck - # - name: reject - # Step 6: Topography Check + + # Step 9: Topography Check - filter: Perform Action filter variables: - name: brightnessTemperature @@ -287,7 +241,7 @@ channels: *{{observation_from_jcb}}_simulated_channels sensor: *{{observation_from_jcb}}_sensor_id - # Step 7: Transmittance Top Check + # Step 10: Transmittance Top Check - filter: Perform Action filter variables: - name: brightnessTemperature @@ -299,7 +253,8 @@ channels: *{{observation_from_jcb}}_simulated_channels options: channels: *{{observation_from_jcb}}_simulated_channels - # Step 8: Cloud Detection Check + + # Step 11: Cloud Detection Check - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -379,11 +334,8 @@ maxvalue: 1.0e-12 action: name: reject - # actions: - # - name: set - # flag: CloudDetection - # - name: reject - # Step 9: NSST Retrieval Check + + # Step 12: NSST Retrieval Check - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -430,11 +382,8 @@ maxvalue: 1.0e-12 action: name: reject - # actions: - # - name: set - # flag: NSSTCheck - # - name: reject - # Step 10: Surface Jacobians Check + + # Step 13: Surface Jacobians Check - filter: Perform Action filter variables: - name: brightnessTemperature @@ -450,7 +399,7 @@ obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] sensor: *{{observation_from_jcb}}_sensor_id - # Step 11: Gross check + # Step 14: Gross check - filter: Background Check filter variables: - name: brightnessTemperature @@ -514,11 +463,8 @@ error parameter vector: *{{observation_from_jcb}}_oberr action: name: reject - # actions: - # - name: set - # flag: GrossCheck - # - name: reject - # Step 12: Useflag Check + + # Step 15: Useflag Check - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -563,11 +509,6 @@ minvalue: 1.0e-12 action: name: reject - # actions: - # - name: set - # flag: UseFlagCheck - # - name: reject - # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- From e0758b6dee275781966cee8947414461d8424356 Mon Sep 17 00:00:00 2001 From: Azadeh Gholoubi <51101867+azadeh-gh@users.noreply.github.com> Date: Wed, 30 Oct 2024 09:06:55 -0400 Subject: [PATCH 057/199] Adding ABI GOES16 and GOES17 yaml.j2 files (#32) [#issue115](https://github.com/NOAA-EMC/JEDI-T2O/issues/115) This PR adds YAML configuration files for ABI-GOES16 and ABI GOES-17 into JEDI, along with thinning and observation quality control. This is the NOAA-EMC/jcb-gdas part of the [#PR1334](https://github.com/NOAA-EMC/GDASApp/pull/1334) **New files include:** - `observation_chronicle/atmosphere/abi_g16.yaml` - `observation_chronicle/atmosphere/abi_g17.yaml` - `observations/atmosphere/abi_g16.yaml.j2` - `observations/atmosphere/abi_g17.yaml.j2` --------- Co-authored-by: Azadeh Gholoubi Co-authored-by: Azadeh Gholoubi Co-authored-by: Dan Holdaway <27729500+danholdaway@users.noreply.github.com> --- .../atmosphere/abi_g16.yaml | 29 ++ .../atmosphere/abi_g17.yaml | 29 ++ .../observations/atmosphere/abi_g16.yaml.j2 | 474 ++++++++++++++++++ .../observations/atmosphere/abi_g17.yaml.j2 | 474 ++++++++++++++++++ 4 files changed, 1006 insertions(+) create mode 100644 parm/jcb-gdas/observation_chronicle/atmosphere/abi_g16.yaml create mode 100644 parm/jcb-gdas/observation_chronicle/atmosphere/abi_g17.yaml create mode 100644 parm/jcb-gdas/observations/atmosphere/abi_g16.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/abi_g17.yaml.j2 diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/abi_g16.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/abi_g16.yaml new file mode 100644 index 000000000..cf76cd889 --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/atmosphere/abi_g16.yaml @@ -0,0 +1,29 @@ +# Instrument metadata +# ------------------- +commissioned: 2017-12-18T00:00:00 +observer_type: satellite # Type of chronicle to use + +# Instrument initial configuration +# -------------------------------- +channel_variables: + simulated: min + active: min + biascorrtd: min + error: max + error_cld: max + ermax: max + +channel_values: +# sim act bcd err errcld ermax + 7: [ 1, -1, -1, 1.40, 0.00, 2.50 ] + 8: [ 1, 1, -1, 3.00, 0.00, 4.00 ] + 9: [ 1, -1, -1, 2.50, 0.00, 4.00 ] + 10: [ 1, 1, -1, 2.20, 0.00, 3.50 ] + 11: [ 1, -1, -1, 2.00, 0.00, 2.50 ] + 12: [ 1, -1, -1, 2.00, 0.00, 2.50 ] + 13: [ 1, -1, -1, 2.00, 0.00, 2.50 ] + 14: [ 1, -1, -1, 2.00, 0.00, 2.50 ] + 15: [ 1, -1, -1, 2.00, 0.00, 2.50 ] + 16: [ 1, -1, -1, 2.00, 0.00, 2.50 ] + + diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/abi_g17.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/abi_g17.yaml new file mode 100644 index 000000000..b891419b9 --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/atmosphere/abi_g17.yaml @@ -0,0 +1,29 @@ +# Instrument metadata +# ------------------- +commissioned: 2019-02-12T00:00:00 +observer_type: satellite # Type of chronicle to use + +# Instrument initial configuration +# -------------------------------- +channel_variables: + simulated: min + active: min + biascorrtd: min + error: max + error_cld: max + ermax: max + +channel_values: +# sim act bcd err errcld ermax + 7: [ 1, -1, -1, 1.40, 0.00, 2.50 ] + 8: [ 1, -1, -1, 3.00, 0.00, 4.00 ] + 9: [ 1, -1, -1, 2.50, 0.00, 4.00 ] + 10: [ 1, -1, -1, 2.20, 0.00, 3.50 ] + 11: [ 1, -1, -1, 2.00, 0.00, 2.50 ] + 12: [ 1, -1, -1, 2.00, 0.00, 2.50 ] + 13: [ 1, -1, -1, 2.00, 0.00, 2.50 ] + 14: [ 1, -1, -1, 2.00, 0.00, 2.50 ] + 15: [ 1, -1, -1, 2.00, 0.00, 2.50 ] + 16: [ 1, -1, -1, 2.00, 0.00, 2.50 ] + + diff --git a/parm/jcb-gdas/observations/atmosphere/abi_g16.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/abi_g16.yaml.j2 new file mode 100644 index 000000000..9d0d0d673 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/abi_g16.yaml.j2 @@ -0,0 +1,474 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: abi_g16 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [brightnessTemperature] + channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} + + # Observation Operator + # -------------------- + obs operator: + name: CRTM + Absorbers: [H2O, O3, CO2] + Clouds: [Water, Ice] + Cloud_Fraction: 1.0 + obs options: + Sensor_ID: &{{observation_from_jcb}}_sensor_id abi_g16 + EndianType: little_endian + CoefficientPath: "{{crtm_coefficient_path}}" + linear obs operator: + Absorbers: [H2O, O3] + + # Observation Bias Correction (VarBC) + # ----------------------------------- + obs bias: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" + variational bc: + predictors: + - name: constant + - name: lapseRate + order: 2 + tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" + - name: lapseRate + tlapse: *{{observation_from_jcb}}_tlapse + - name: emissivityJacobian + - name: sensorScanAngle + var_name: sensorScanPosition + order: 4 + - name: sensorScanAngle + var_name: sensorScanPosition + order: 3 + - name: sensorScanAngle + var_name: sensorScanPosition + order: 2 + - name: sensorScanAngle + var_name: sensorScanPosition + covariance: + minimal required obs number: 20 + variance range: [1.0e-6, 10.0] + step size: 1.0e-4 + largest analysis variance: 10000.0 + prior: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiascovin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovin_suffix}}" + inflation: + ratio: 1.1 + ratio for small dataset: 2.0 + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" + + # Observation Filters (QC) + # ------------------------ + obs prior filters: + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: assign error + error parameter vector: &{{observation_from_jcb}}_err [{{ get_satellite_variable(observation_from_jcb, "error") }}] + + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/surfaceFlag + type: int + function: + name: IntObsFunction/Conditional + options: + defaultvalue: 4 + firstmatchingcase: true + cases: + - where: + - variable: + name: GeoVaLs/water_area_fraction + minvalue: 0.99 + value: 0 + - where: + - variable: + name: GeoVaLs/land_area_fraction + minvalue: 0.99 + value: 1 + - where: + - variable: + name: GeoVaLs/ice_area_fraction + minvalue: 0.99 + value: 2 + - where: + - variable: + name: GeoVaLs/surface_snow_area_fraction + minvalue: 0.99 + value: 3 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/surfaceParam + type: int + function: + name: IntObsFunction/Conditional + options: + defaultvalue: 0 + firstmatchingcase: true + cases: + - where: + - variable: + name: GeoVaLs/water_area_fraction + minvalue: 0.99 + value: 30 + - where: + - variable: + name: GeoVaLs/land_area_fraction + minvalue: 0.99 + value: 15 + - where: + - variable: + name: GeoVaLs/ice_area_fraction + minvalue: 0.99 + value: 20 + - where: + - variable: + name: GeoVaLs/surface_snow_area_fraction + minvalue: 0.99 + value: 15 + + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/thinningCriteria + type: int + function: + name: IntObsFunction/Arithmetic + options: + variables: + - name: DerivedMetaData/surfaceParam + coefs: [1] + + obs post filters: + # Satellite Zenith Angle Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: MetaData/sensorZenithAngle + maxvalue: 65. + action: + name: reject + +# from read_abi: Reject when cloud fraction is more than 30 + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: 7-16 + where: + - variable: + name: MetaData/cloudAmount + channels: 8 + maxvalue: 30. + +# from read_abi: WV Channels 8-10 scene consistency check SDTB>1.3 + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: 7-16 + where: + - variable: + name: ClearSkyStdDev/brightnessTemperature + channels: 8-10 + maxvalue: 1.3 + max_exclusive: true + +# Thinning + - filter: Gaussian Thinning + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + horizontal_mesh: 145 + use_reduced_horizontal_grid: true + distance_norm: maximum + time_mesh: 'PT06H' + time_min: '2021-07-31T21:00:00Z' + time_max: '2021-08-01T03:00:00Z' + priority_variable: DerivedMetaData/thinningCriteria + action: + name: reject + +# Surface type check + - filter: RejectList + filter variables: + - name: brightnessTemperature + channels: 7, 11-16 + where: + - variable: + name: GeoVaLs/land_area_fraction + minvalue: 0.99 + + - filter: RejectList + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: GeoVaLs/surface_snow_area_fraction + minvalue: 0.99 + + - filter: RejectList + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: GeoVaLs/ice_area_fraction + minvalue: 0.99 + + - filter: RejectList + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: GeoVaLs/land_area_fraction + maxvalue: 0.99 + max_exclusive: true + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + max_exclusive: true + - variable: + name: GeoVaLs/ice_area_fraction + maxvalue: 0.99 + max_exclusive: true + - variable: + name: GeoVaLs/surface_snow_area_fraction + maxvalue: 0.99 + max_exclusive: true + +# BT range check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + minvalue: 0.0 + maxvalue: 1000.0 + action: + name: reject + +# Model top transmittance check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorTransmitTopRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + +# Cloud detection + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/CloudDetectMinResidualIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: &{{observation_from_jcb}}_active_channels [{{ get_satellite_variable(observation_from_jcb, "active") }}] + use_flag_clddet: [-2, -2, -2, -2, -2, -2, -2, -2, -2, -2] + obserr_dtempf: [0.5, 2.0, 4.0, 2.0, 4.0] + error parameter vector: *{{observation_from_jcb}}_err + maxvalue: 1.0e-12 + action: + name: reject + +# Scene Consistency Check based on ch13 (10.8 micron) + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: 7, 10-16 + where: + - variable: + name: ClearSkyStdDev/brightnessTemperature + channels: 13 + maxvalue: 0.5 + max_exclusive: true + + +# Scene Consistency Channels 8-10 Based on ch 8 SDTB + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: 8-10 + where: + - variable: + name: ClearSkyStdDev/brightnessTemperature_8 + maxvalue: 0.5 + minvalue: 0.4 + min_exclusive: true + action: + name: inflate error + inflation factor: 1.1489 + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: 8-10 + where: + - variable: + name: ClearSkyStdDev/brightnessTemperature_8 + maxvalue: 0.6 +# max_exclusive: true + minvalue: 0.5 + min_exclusive: true + action: + name: inflate error + inflation factor: 1.2923 + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: 8-10 + where: + - variable: + name: ClearSkyStdDev/brightnessTemperature_8 + maxvalue: 0.7 +# max_exclusive: true + minvalue: 0.6 + min_exclusive: true + action: + name: inflate error + inflation factor: 1.4967 + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: 8-10 + where: + - variable: + name: ClearSkyStdDev/brightnessTemperature_8 + minvalue: 0.7 + min_exclusive: true + action: + name: inflate error + inflation factor: 1.5199 + +# Near SST Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/NearSSTRetCheckIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: *{{observation_from_jcb}}_active_channels + maxvalue: 1.0e-12 + action: + name: reject + +# Surface Jacobian + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorSurfJacobianRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + sensor: *{{observation_from_jcb}}_sensor_id + obserr_demisf: [0.01, 0.02, 0.03, 0.02, 0.03] + obserr_dtempf: [0.5, 2.0, 4.0, 2.0, 4.0] + +# Channels 7, 10-16 Cloud fraction >2 + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: 7, 10-16 + where: + - variable: + name: MetaData/cloudAmount + maxvalue: 2 + +# Variable assignment, OmF non bias corrected + - filter: Variable Assignment + assignments: + - name: OmFNBC@DerivedMetaData + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: brightnessTemperature@ObsValue + channels: 13 + - name: brightnessTemperature@HofX + channels: 13 + - name: brightnessTemperature@ObsBiasData + channels: 13 + - name: brightnessTemperature@ObsValue + channels: 14 + - name: brightnessTemperature@HofX + channels: 14 + - name: brightnessTemperature@ObsBiasData + channels: 14 + coefs: [1, -1, 1, -1, 1, -1] + +# Channels 7, 10-16, OmFNBC < -0.75 + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: 7, 10-16 + where: + - variable: + name: OmFNBC@DerivedMetaData + minvalue: -0.75 + max_exclusive: true + +# Final gross check + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + function absolute threshold: + - name: ObsFunction/ObsErrorBoundIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_latitude: + name: ObsFunction/ObsErrorFactorLatRad + options: + latitude_parameters: [0.0, 0.0, 0.0, 0.0] + obserr_bound_transmittop: + name: ObsFunction/ObsErrorFactorTransmitTopRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_max: [{{ get_satellite_variable(observation_from_jcb, "ermax") }}] + error parameter vector: *{{observation_from_jcb}}_err + action: + name: reject + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 + diff --git a/parm/jcb-gdas/observations/atmosphere/abi_g17.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/abi_g17.yaml.j2 new file mode 100644 index 000000000..dc8145bae --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/abi_g17.yaml.j2 @@ -0,0 +1,474 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: abi_g17 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [brightnessTemperature] + channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} + + # Observation Operator + # -------------------- + obs operator: + name: CRTM + Absorbers: [H2O, O3, CO2] + Clouds: [Water, Ice] + Cloud_Fraction: 1.0 + obs options: + Sensor_ID: &{{observation_from_jcb}}_sensor_id abi_g17 + EndianType: little_endian + CoefficientPath: "{{crtm_coefficient_path}}" + linear obs operator: + Absorbers: [H2O, O3] + + # Observation Bias Correction (VarBC) + # ----------------------------------- + obs bias: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" + variational bc: + predictors: + - name: constant + - name: lapseRate + order: 2 + tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" + - name: lapseRate + tlapse: *{{observation_from_jcb}}_tlapse + - name: emissivityJacobian + - name: sensorScanAngle + var_name: sensorScanPosition + order: 4 + - name: sensorScanAngle + var_name: sensorScanPosition + order: 3 + - name: sensorScanAngle + var_name: sensorScanPosition + order: 2 + - name: sensorScanAngle + var_name: sensorScanPosition + covariance: + minimal required obs number: 20 + variance range: [1.0e-6, 10.0] + step size: 1.0e-4 + largest analysis variance: 10000.0 + prior: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiascovin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovin_suffix}}" + inflation: + ratio: 1.1 + ratio for small dataset: 2.0 + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" + + # Observation Filters (QC) + # ------------------------ + obs prior filters: + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: assign error + error parameter vector: &{{observation_from_jcb}}_err [{{ get_satellite_variable(observation_from_jcb, "error") }}] + + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/surfaceFlag + type: int + function: + name: IntObsFunction/Conditional + options: + defaultvalue: 4 + firstmatchingcase: true + cases: + - where: + - variable: + name: GeoVaLs/water_area_fraction + minvalue: 0.99 + value: 0 + - where: + - variable: + name: GeoVaLs/land_area_fraction + minvalue: 0.99 + value: 1 + - where: + - variable: + name: GeoVaLs/ice_area_fraction + minvalue: 0.99 + value: 2 + - where: + - variable: + name: GeoVaLs/surface_snow_area_fraction + minvalue: 0.99 + value: 3 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/surfaceParam + type: int + function: + name: IntObsFunction/Conditional + options: + defaultvalue: 0 + firstmatchingcase: true + cases: + - where: + - variable: + name: GeoVaLs/water_area_fraction + minvalue: 0.99 + value: 30 + - where: + - variable: + name: GeoVaLs/land_area_fraction + minvalue: 0.99 + value: 15 + - where: + - variable: + name: GeoVaLs/ice_area_fraction + minvalue: 0.99 + value: 20 + - where: + - variable: + name: GeoVaLs/surface_snow_area_fraction + minvalue: 0.99 + value: 15 + + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/thinningCriteria + type: int + function: + name: IntObsFunction/Arithmetic + options: + variables: + - name: DerivedMetaData/surfaceParam + coefs: [1] + + obs post filters: + # Satellite Zenith Angle Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: MetaData/sensorZenithAngle + maxvalue: 65. + action: + name: reject + +# from read_abi: Reject when cloud fraction is more than 30 + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: 7-16 + where: + - variable: + name: MetaData/cloudAmount + channels: 8 + maxvalue: 30. + +# from read_abi: WV Channels 8-10 scene consistency check SDTB>1.3 + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: 7-16 + where: + - variable: + name: ClearSkyStdDev/brightnessTemperature + channels: 8-10 + maxvalue: 1.3 + max_exclusive: true + +# Thinning + - filter: Gaussian Thinning + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + horizontal_mesh: 145 + use_reduced_horizontal_grid: true + distance_norm: maximum + time_mesh: 'PT06H' + time_min: '2021-07-31T21:00:00Z' + time_max: '2021-08-01T03:00:00Z' + priority_variable: DerivedMetaData/thinningCriteria + action: + name: reject + +# Surface type check + - filter: RejectList + filter variables: + - name: brightnessTemperature + channels: 7, 11-16 + where: + - variable: + name: GeoVaLs/land_area_fraction + minvalue: 0.99 + + - filter: RejectList + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: GeoVaLs/surface_snow_area_fraction + minvalue: 0.99 + + - filter: RejectList + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: GeoVaLs/ice_area_fraction + minvalue: 0.99 + + - filter: RejectList + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: GeoVaLs/land_area_fraction + maxvalue: 0.99 + max_exclusive: true + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + max_exclusive: true + - variable: + name: GeoVaLs/ice_area_fraction + maxvalue: 0.99 + max_exclusive: true + - variable: + name: GeoVaLs/surface_snow_area_fraction + maxvalue: 0.99 + max_exclusive: true + +# BT range check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + minvalue: 0.0 + maxvalue: 1000.0 + action: + name: reject + +# Model top transmittance check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorTransmitTopRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + +# Cloud detection + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/CloudDetectMinResidualIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: &{{observation_from_jcb}}_active_channels [{{ get_satellite_variable(observation_from_jcb, "active") }}] + use_flag_clddet: [-2, -2, -2, -2, -2, -2, -2, -2, -2, -2] + obserr_dtempf: [0.5, 2.0, 4.0, 2.0, 4.0] + error parameter vector: *{{observation_from_jcb}}_err + maxvalue: 1.0e-12 + action: + name: reject + +# Scene Consistency Check based on ch13 (10.8 micron) + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: 7, 10-16 + where: + - variable: + name: ClearSkyStdDev/brightnessTemperature + channels: 13 + maxvalue: 0.5 + max_exclusive: true + + +# Scene Consistency Channels 8-10 Based on ch 8 SDTB + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: 8-10 + where: + - variable: + name: ClearSkyStdDev/brightnessTemperature_8 + maxvalue: 0.5 + minvalue: 0.4 + min_exclusive: true + action: + name: inflate error + inflation factor: 1.1489 + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: 8-10 + where: + - variable: + name: ClearSkyStdDev/brightnessTemperature_8 + maxvalue: 0.6 +# max_exclusive: true + minvalue: 0.5 + min_exclusive: true + action: + name: inflate error + inflation factor: 1.2923 + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: 8-10 + where: + - variable: + name: ClearSkyStdDev/brightnessTemperature_8 + maxvalue: 0.7 +# max_exclusive: true + minvalue: 0.6 + min_exclusive: true + action: + name: inflate error + inflation factor: 1.4967 + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: 8-10 + where: + - variable: + name: ClearSkyStdDev/brightnessTemperature_8 + minvalue: 0.7 + min_exclusive: true + action: + name: inflate error + inflation factor: 1.5199 + +# Near SST Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/NearSSTRetCheckIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: *{{observation_from_jcb}}_active_channels + maxvalue: 1.0e-12 + action: + name: reject + +# Surface Jacobian + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorSurfJacobianRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + sensor: *{{observation_from_jcb}}_sensor_id + obserr_demisf: [0.01, 0.02, 0.03, 0.02, 0.03] + obserr_dtempf: [0.5, 2.0, 4.0, 2.0, 4.0] + +# Channels 7, 10-16 Cloud fraction >2 + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: 7, 10-16 + where: + - variable: + name: MetaData/cloudAmount + maxvalue: 2 + +# Variable assignment, OmF non bias corrected + - filter: Variable Assignment + assignments: + - name: OmFNBC@DerivedMetaData + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: brightnessTemperature@ObsValue + channels: 13 + - name: brightnessTemperature@HofX + channels: 13 + - name: brightnessTemperature@ObsBiasData + channels: 13 + - name: brightnessTemperature@ObsValue + channels: 14 + - name: brightnessTemperature@HofX + channels: 14 + - name: brightnessTemperature@ObsBiasData + channels: 14 + coefs: [1, -1, 1, -1, 1, -1] + +# Channels 7, 10-16, OmFNBC < -0.75 + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: 7, 10-16 + where: + - variable: + name: OmFNBC@DerivedMetaData + minvalue: -0.75 + max_exclusive: true + +# Final gross check + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + function absolute threshold: + - name: ObsFunction/ObsErrorBoundIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_latitude: + name: ObsFunction/ObsErrorFactorLatRad + options: + latitude_parameters: [0.0, 0.0, 0.0, 0.0] + obserr_bound_transmittop: + name: ObsFunction/ObsErrorFactorTransmitTopRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_max: [{{ get_satellite_variable(observation_from_jcb, "ermax") }}] + error parameter vector: *{{observation_from_jcb}}_err + action: + name: reject + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 + From 011a2a5789b0c3ab2c6031257c35e376e68d214e Mon Sep 17 00:00:00 2001 From: Dan Holdaway <27729500+danholdaway@users.noreply.github.com> Date: Wed, 30 Oct 2024 11:14:49 -0400 Subject: [PATCH 058/199] Move test base YAMLs from jcb to client (#38) This brings the test YAMLs in from JCB to eliminate the need for synced PRs all the time. Co-authored-by: danholdaway --- .../gdas-atmosphere-templates.yaml | 136 ++++++++++++++++++ .../gdas-marine-templates.yaml | 111 ++++++++++++++ 2 files changed, 247 insertions(+) create mode 100644 parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml create mode 100644 parm/jcb-gdas/test/client_integration/gdas-marine-templates.yaml diff --git a/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml b/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml new file mode 100644 index 000000000..bea2b27fb --- /dev/null +++ b/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml @@ -0,0 +1,136 @@ +# This part is for testing only. Normally this would just show algorithm: +# ----------------------------------------------------------------------------------- +supported_algorithms: +- 3dvar +- hofx3d +- hofx4d +- local_ensemble_da +# Local algorithms +- fv3jedi_fv3inc_lgetkf +- fv3jedi_fv3inc_variational + +# This is overwritten in the testing but allows command line test of jcb +algorithm: fv3jedi_fv3inc_lgetkf + +# Search path for model and obs for JCB (relative for the submodule, or can be absolute) +# -------------------------------------------------------------------------------------- +app_path_algorithm: gdas/algorithm/atmosphere +app_path_model: gdas/model/atmosphere +app_path_observations: gdas/observations/atmosphere +app_path_observation_chronicle: gdas/observation_chronicle/atmosphere + + +# Places where we deviate from the generic file name of a yaml +# ------------------------------------------------------------ +final_increment_file: atmosphere_final_increment_gaussian +output_ensemble_increments_file: atmosphere_output_ensemble_increments_gaussian +model_file: atmosphere_model_pseudo +initial_condition_file: atmosphere_background +background_error_file: atmosphere_background_error_hybrid_gsibec_bump + + +# Global analysis things +# ---------------------- +window_begin: '2024-02-01T00:00:00Z' +window_length: PT6H +bound_to_include: begin +minimizer: DRPCG +final_diagnostics_departures: anlmob +analysis_variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] +number_of_outer_loops: 2 + + +# Model things +# ------------ +atmosphere_layout_x: 2 +atmosphere_layout_y: 2 +atmosphere_npx_ges: 361 +atmosphere_npy_ges: 361 +atmosphere_npz_ges: 127 +atmosphere_npx_anl: 361 +atmosphere_npy_anl: 361 +atmosphere_npz_anl: 127 +atmosphere_fv3jedi_files_path: DATA/fv3jedi + +# Background +atmosphere_background_path: DATA/bkg +atmosphere_background_ensemble_path: "DATA/ens/mem%mem%" + +atmosphere_variational_history_prefix: "bkg_" +atmosphere_ensemble_history_prefix: "ens_" + +atmosphere_background_time_iso: '2024-02-02T00:00:00Z' + +# Background error + +atmosphere_bump_data_directory: DATA/berror +atmosphere_gsibec_path: DATA/berror +atmosphere_number_ensemble_members: 3 +atmosphere_layout_gsib_x: 2 +atmosphere_layout_gsib_y: 2 + + +# Forecasting +atmosphere_forecast_length: PT6H +atmosphere_forecast_timestep: PT1H + +# Write final increment on Gaussian grid in variational +atmosphere_final_increment_prefix: "./anl/atminc." + + +# Observation things +# ------------------ +observations: all_observations + +crtm_coefficient_path: "DATA/crtm/" + +# Naming conventions for observational files +atmosphere_obsdatain_path: DATA/obs +atmosphere_obsdatain_prefix: OPREFIX +atmosphere_obsdatain_suffix: ".2024020100.nc" + +atmosphere_obsdataout_path: DATA/diags +atmosphere_obsdataout_prefix: diag +atmosphere_obsdataout_suffix: "_2024020100.nc" + +# Naming conventions for bias correction files +atmosphere_obsbiasin_path: DATA/obs +atmosphere_obsbiasin_prefix: GPREFIX +atmosphere_obsbiasin_suffix: ".satbias.nc" +atmosphere_obstlapsein_prefix: GPREFIX +atmosphere_obstlapsein_suffix: ".tlapse.txt" +atmosphere_obsbiascovin_prefix: GPREFIX +atmosphere_obsbiascovin_suffix: ".satbias_cov.nc" + +atmosphere_obsbiasout_path: DATA/bc +atmosphere_obsbiasout_prefix: APREFIX +atmosphere_obsbiasout_suffix: ".satbias.nc" +atmosphere_obsbiascovout_prefix: APREFIX +atmosphere_obsbiascovout_suffix: ".satbias_cov.nc" + + +# Local Ensemble DA (LETKF) +# ------------------------- +local_ensemble_da_solver: GETKF + +increment_variables: [ua,va,DZ,delp,t,ps,sphum,ice_wat,liq_wat,o3mr] + +# Veritcal localization for GETKF +vl_fraction_of_retained_variance: 0.750 +vl_lengthscale: 2.1 +vl_lengthscale_units: logp +inflation_rtps: 0.85 +inflation_rtpp: 0.0 +inflation_mult: 1.0 + +# Driver +driver_update_obs_config_with_geometry_info: true +driver_save_posterior_mean: false +driver_save_posterior_ensemble: false +driver_save_prior_mean: false +driver_save_posterior_mean_increment: false +driver_save_posterior_ensemble_increments: true + +# Diagnostics +atmosphere_ensemble_increment_prefix: "./anl/mem%{member}%/atminc." +atmosphere_posterior_output_gaussian: "./mem%{member}%/atmanl." diff --git a/parm/jcb-gdas/test/client_integration/gdas-marine-templates.yaml b/parm/jcb-gdas/test/client_integration/gdas-marine-templates.yaml new file mode 100644 index 000000000..2fe3e797a --- /dev/null +++ b/parm/jcb-gdas/test/client_integration/gdas-marine-templates.yaml @@ -0,0 +1,111 @@ +# This part is for testing only. Normally this would just show algorithm: +# ----------------------------------------------------------------------------------- +supported_algorithms: +- 3dvar +- hofx3d +- hofx4d +- 3dfgat +#- local_ensemble_da + + +# Search path for model and obs for JCB (relative for the submodule, or can be absolute) +# -------------------------------------------------------------------------------------- +app_path_model: gdas/model/marine +app_path_observations: gdas/observations/marine +app_path_observation_chronicle: gdas/observation_chronicle/marine + + +# Places where we deviate from the generic file name of a yaml +# ------------------------------------------------------------ +model_file: marine_model_pseudo +initial_condition_file: marine_background +background_error_file: marine_background_error_hybrid_diffusion_diffusion + + +# Global analysis things +# ---------------------- +window_begin: '2024-01-31T21:00:00Z' +window_length: PT6H +bound_to_include: begin +minimizer: RPCG +final_diagnostics_departures: oman +analysis_variables: [cicen, hicen, hsnon, socn, tocn, uocn, vocn, ssh] +number_of_outer_loops: 1 + + +# Model things +# ------------ +marine_soca_files_path: DATA/soca + +# Background +marine_background_time: '2024-01-31T21:00:00Z' +marine_background_path: DATA/bkg + + +# Background error +marine_window_middle: '2024-02-01T00:00:00Z' +marine_number_ensemble_members: 3 + +# Forecasting +marine_forecast_timestep: PT1H + +# Minimization +marine_ninner_1: 3 +marine_grad_red_1: 1e-4 + +# Forecast states +marine_pseudo_model_states: +- date: '2024-01-31T21:00:00Z' + basename: base + ocn_filename: ocn + ice_filename: ice + read_from_file: true +- date: '2024-02-01T00:00:00Z' + basename: base + ocn_filename: ocn + ice_filename: ice + read_from_file: true +- date: '2024-02-01T03:00:00Z' + basename: base + ocn_filename: ocn + ice_filename: ice + read_from_file: true + +# Observation things +# ------------------ +observations: all_observations + +# Naming conventions for observational files +marine_obsdatain_path: DATA/obs +marine_obsdatain_prefix: OPREFIX +marine_obsdatain_suffix: ".2024020100.nc" + +marine_obsdataout_path: DATA/diags +marine_obsdataout_prefix: diag +marine_obsdataout_suffix: "_2024020100.nc" + +# Local Ensemble DA (LETKF) +# ------------------------- +local_ensemble_da_solver: GETKF + +increment_variables: [cicen, hicen, hsnon, socn, tocn, uocn, vocn, ssh] + +# Veritcal localization for GETKF +vl_fraction_of_retained_variance: 0.750 +vl_lengthscale: 2.1 +vl_lengthscale_units: logp +inflation_rtps: 0.85 +inflation_rtpp: 0.0 +inflation_mult: 1.0 + +# Driver +driver_update_obs_config_with_geometry_info: true +driver_save_posterior_mean: false +driver_save_posterior_ensemble: false +driver_save_prior_mean: false +driver_save_posterior_mean_increment: false +driver_save_posterior_ensemble_increments: true + +# Diagnostics +marine_ensemble_increment_prefix: "./anl/mem%{member}%/atminc." +marine_posterior_output_gaussian: "./mem%{member}%/atmanl." From 1940a117beda353db2ca8bc75c1c587341689a09 Mon Sep 17 00:00:00 2001 From: Guillaume Vernieres Date: Wed, 30 Oct 2024 12:09:46 -0400 Subject: [PATCH 059/199] Marine ens. var. (#37) Companion PR to https://github.com/NOAA-EMC/GDASApp/pull/1353 --------- Co-authored-by: Dan Holdaway <27729500+danholdaway@users.noreply.github.com> --- ...d_error_hybrid_diffusion_diffusion.yaml.j2 | 67 +-- .../marine/insitu_profile_argo.yaml.j2 | 542 ------------------ .../gdas-marine-templates.yaml | 2 + 3 files changed, 24 insertions(+), 587 deletions(-) diff --git a/parm/jcb-gdas/model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 b/parm/jcb-gdas/model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 index 43feedb7b..b2709aafb 100644 --- a/parm/jcb-gdas/model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 +++ b/parm/jcb-gdas/model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 @@ -3,39 +3,27 @@ components: - covariance: covariance model: SABER saber central block: - saber block name: EXPLICIT_DIFFUSION - active variables: [tocn, socn, ssh, cicen] - geometry: - mom6_input_nml: mom_input.nml - fields metadata: ./fields_metadata.yaml - group mapping: - - name: ocean - variables: - - tocn - - socn - - ssh - - name: ice - variables: - - cicen + saber block name: diffusion read: groups: - - name: ocean + - variables: [tocn, socn, ssh] horizontal: - filename: hz_ocean.nc + filepath: ./staticb/hz_ocean vertical: - filename: vt_ocean.nc - - name: ice + levels: {{marine_vt_levels}} + filepath: ./staticb/vt_ocean + - variables: [cicen] horizontal: - filename: hz_ice.nc + filepath: ./staticb/hz_ice saber outer blocks: - saber block name: StdDev read: model file: - date: '{{marine_window_middle}}' - basename: ./ - ocn_filename: 'ocn.bkgerr_stddev.incr.{{marine_window_middle}}.nc' - ice_filename: 'ice.bkgerr_stddev.incr.{{marine_window_middle}}.nc' + date: '{{marine_stddev_time}}' + basename: ./staticb/ + ocn_filename: 'ocn.bkgerr_stddev.nc' + ice_filename: 'ice.bkgerr_stddev.nc' read_from_file: 3 linear variable change: @@ -43,7 +31,6 @@ components: output variables: [cicen, hicen, hsnon, socn, tocn, uocn, vocn, ssh] linear variable changes: - linear variable change name: BalanceSOCA - weight: value: 1.00 @@ -52,41 +39,31 @@ components: members from template: template: read_from_file: 1 - date: '{{marine_window_middle}}' - basename: ./static_ens/ + date: '{{marine_stddev_time}}' + basename: ../ensdata/ ocn_filename: 'ocn.pert.steric.%mem%.nc' - ice_filename: 'ice.%mem%.nc' + ice_filename: 'ens/ice.%mem%.nc' state variables: [tocn, socn, ssh, uocn, vocn, cicen, hicen, hsnon] pattern: '%mem%' nmembers: '{{marine_number_ensemble_members}}' localization: localization method: SABER saber central block: - saber block name: EXPLICIT_DIFFUSION - active variables: [tocn, socn, ssh] - geometry: - mom6_input_nml: mom_input.nml - fields metadata: ./fields_metadata.yaml - group mapping: - - name: ocean - variables: [tocn, socn, ssh, uocn, vocn] - - name: ice - variables: [cicen, hicen, hsnon] + saber block name: diffusion read: groups: - - name: ocean + - variables: [tocn, socn, ssh] multivariate strategy: duplicated horizontal: - filename: hz_ocean.nc + filepath: ./staticb/hz_ocean vertical: strategy: duplicated - - name: ice + - variables: [cicen] horizontal: - filename: hz_ice.nc - + filepath: ./staticb/hz_ice weight: read_from_file: 3 basename: ./ - ocn_filename: 'ocn.ens_weights.incr.{{marine_window_middle}}.nc' - ice_filename: 'ice.ens_weights.incr.{{marine_window_middle}}.nc' - date: '{{marine_window_middle}}' + ocn_filename: 'ocean.ens_weights.nc' + ice_filename: 'ice.ens_weights.nc' + date: '{{marine_stddev_time}}' diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_argo.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_argo.yaml.j2 index 4a1dbd7bc..0d121a013 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_argo.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_argo.yaml.j2 @@ -15,7 +15,6 @@ obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" simulated variables: [waterTemperature, salinity] observed variables: [waterTemperature, salinity] - derived variables: [waterPressure] io pool: max pool size: 1 obs operator: @@ -60,544 +59,3 @@ minvalue: -2.5 maxvalue: 40.0 - #----------------------------------------------------------------------------- - ### Regional range tests - #----------------------------------------------------------------------------- - - #### Red Sea - #----------------------------------------------------------------------------- - #### - #### the document linked here describes Red sea as the are between 10N, 40E; - #### 20N, 50E; 30N, 30E; 10N, 40E. But that would also include Gulf of Aden. - #### A more reasonable choice seemed to be a box that extends from 10 N to - #### 30 N and 30 E to 45 East . - - - filter: Bounds Check - filter variables: [{name: waterTemperature}] - minvalue: 21.7 - maxvalue: 40.0 - where: - - variable: - name: MetaData/latitude - minvalue: 10 - maxvalue: 30 - - variable: - name: MetaData/longitude - minvalue: 30 - maxvalue: 45 - - #### Mediterranean Sea - #----------------------------------------------------------------------------- - ##### Area 1/3 for Mediterranean Sea - - filter: Bounds Check - filter variables: [{name: waterTemperature}] - minvalue: 10.0 - maxvalue: 40.0 - where: - - variable: - name: MetaData/latitude - minvalue: 30 - maxvalue: 40 - - variable: - name: MetaData/longitude - minvalue: -6 - maxvalue: 40 - ##### Area 2/3 for Mediterranean Sea - - filter: Bounds Check - filter variables: [{name: waterTemperature}] - minvalue: 10.0 - maxvalue: 40.0 - where: - - variable: - name: MetaData/latitude - minvalue: 40 - maxvalue: 41.5 - - variable: - name: MetaData/longitude - minvalue: 20 - maxvalue: 30 - ##### Area 3/3 for Mediterranean Sea - - filter: Bounds Check - filter variables: [{name: waterTemperature}] - minvalue: 10.0 - maxvalue: 40.0 - where: - - variable: - name: MetaData/latitude - minvalue: 40 - maxvalue: 46 - - variable: - name: MetaData/longitude - minvalue: 0 - maxvalue: 20 - - #### Northwestern shelves - #----------------------------------------------------------------------------- - - filter: Bounds Check - filter variables: [{name: waterTemperature}] - minvalue: -2.0 - maxvalue: 24.0 - where: - - variable: - name: MetaData/latitude - minvalue: 50 - maxvalue: 60 - - variable: - name: MetaData/longitude - minvalue: -20 - maxvalue: 10 - #### Southwestern shelves - #----------------------------------------------------------------------------- - - filter: Bounds Check - filter variables: [{name: waterTemperature}] - minvalue: -2.0 - maxvalue: 30 - where: - - variable: - name: MetaData/latitude - minvalue: 25 - maxvalue: 50 - - variable: - name: MetaData/longitude - minvalue: -30 - maxvalue: 0 - - #### Arctic Sea - #----------------------------------------------------------------------------- - - filter: Bounds Check - filter variables: [{name: waterTemperature}] - minvalue: -1.92 - maxvalue: 25.0 - where: - - variable: - name: MetaData/latitude - minvalue: 60 - - - filter: Background Check - filter variables: [{name: waterTemperature}] - threshold: 5.0 - absolute threshold: 5.0 - - #------------------------------------------------------------------------------- - ### Spike test - #----------------------------------------------------------------------------- - - filter: Create Diagnostic Flags - filter variables: - - name: waterTemperature - - name: salinity - flags: - - name: spike - initial value: false - - name: step - initial value: false - - - filter: Spike and Step Check - filter variables: - - name: ObsValue/waterTemperature - dependent: ObsValue/waterTemperature # dy/ - independent: MetaData/depth # dx - count spikes: true - count steps: true - tolerance: - nominal value: 10 # K nominal, in the case of temperature (not really) - gradient: 0.1 # K/m - if dy/dx greater, could be a spike - gradient x resolution: 10 # m - can't know dx to better precision - factors: [1,1,0.5] # multiply tolerance, for ranges bounded by... - x boundaries: [0,500,500] # ...these values of x (depth in m) - boundary layer: - x range: [0.0,300.0] # when bounded by these x values (depth in m)... - step tolerance range: [0.0,-2.0] # ...relax tolerance for steps in boundary layer... - maximum x interval: [50.0,100.0] # ...and ignore level if dx greater than this - action: - name: reject - - #### Count spikes - #----------------------------------------------------------------------------- - - filter: Variable Assignment # create derived obs value containing only T spikes - assignments: - - name: DerivedMetaData/waterTemperature_spikes - type: int - function: - name: IntObsFunction/ProfileLevelCount - options: - where: - - variable: - name: DiagnosticFlags/spike/waterTemperature - value: is_true - - #### Count steps - #----------------------------------------------------------------------------- - - filter: Variable Assignment # create derived obs value containing only T steps - assignments: - - name: DerivedMetaData/waterTemperature_steps - type: int - function: - name: IntObsFunction/ProfileLevelCount - options: - where: - - variable: - name: DiagnosticFlags/step/waterTemperature - value: is_true - #### Count total rejections - #----------------------------------------------------------------------------- - - filter: Variable Assignment # compute sum 2*spikes+steps - assignments: - - name: DerivedMetaData/waterTemperature_rejections - type: int - function: - name: IntObsFunction/LinearCombination - options: - variables: [DerivedMetaData/waterTemperature_spikes, DerivedMetaData/waterTemperature_steps] - coefs: [2,1] - #### Reject entire profile if total rejctions > threshold - #----------------------------------------------------------------------------- - - filter: Perform Action # reject whole profile if 2*spikes+steps>=rejection threshold - where: - - variable: - name: DerivedMetaData/waterTemperature_rejections - minvalue: 3 - action: - name: reject - - #------------------------------------------------------------------------------- - ## Filters for S: - #------------------------------------------------------------------------------- - #----------------------------------------------------------------------------- - ### Global range test - #----------------------------------------------------------------------------- - - filter: Bounds Check - filter variables: [{name: salinity}] - minvalue: 2.0 - maxvalue: 41.0 - - #----------------------------------------------------------------------------- - ### Regional range test - #----------------------------------------------------------------------------- - #### Red Sea - #----------------------------------------------------------------------------- - #### - #### the document linked here describes Red sea as the are between 10N, 40E; - #### 20N, 50E; 30N, 30E; 10N, 40E. But that would also include Gulf of Aden. - #### A more reasonable choice seemed to be a box that extends from 10 N to - #### 30 N and 30 E to 45 East . - - - filter: Bounds Check - filter variables: [{name: salinity}] - minvalue: 2.0 - maxvalue: 41.0 - where: - - variable: - name: MetaData/latitude - minvalue: 10 - maxvalue: 30 - - variable: - name: MetaData/longitude - minvalue: 30 - maxvalue: 45 - - #### Mediterranean Sea - #----------------------------------------------------------------------------- - ##### Area 1/3 for Mediterranean Sea - - filter: Bounds Check - filter variables: [{name: salinity}] - minvalue: 2.0 - maxvalue: 40.0 - where: - - variable: - name: MetaData/latitude - minvalue: 30 - maxvalue: 40 - - variable: - name: MetaData/longitude - minvalue: -6 - maxvalue: 40 - ##### Area 2/3 for Mediterranean Sea - - filter: Bounds Check - filter variables: [{name: salinity}] - minvalue: 2.0 - maxvalue: 40.0 - where: - - variable: - name: MetaData/latitude - minvalue: 40 - maxvalue: 41.5 - - variable: - name: MetaData/longitude - minvalue: 20 - maxvalue: 30 - ##### Area 3/3 for Mediterranean Sea - - filter: Bounds Check - filter variables: [{name: salinity}] - minvalue: 2.0 - maxvalue: 40.0 - where: - - variable: - name: MetaData/latitude - minvalue: 40 - maxvalue: 46 - - variable: - name: MetaData/longitude - minvalue: 0 - maxvalue: 20 - - - #### Northwestern shelves - #----------------------------------------------------------------------------- - - filter: Bounds Check - filter variables: [{name: salinity}] - minvalue: 0.0 - maxvalue: 37.0 - where: - - variable: - name: MetaData/latitude - minvalue: 50 - maxvalue: 60 - - variable: - name: MetaData/longitude - minvalue: -20 - maxvalue: 10 - - #### Southwestern shelves - #----------------------------------------------------------------------------- - - filter: Bounds Check - filter variables: [{name: salinity}] - minvalue: 0.0 - maxvalue: 38 - where: - - variable: - name: MetaData/latitude - minvalue: 25 - maxvalue: 50 - - variable: - name: MetaData/longitude - minvalue: -30 - maxvalue: 0 - - #### Arctic Sea - #----------------------------------------------------------------------------- - - filter: Bounds Check - filter variables: [{name: salinity}] - minvalue: 2.0 - maxvalue: 40.0 - where: - - variable: - name: MetaData/latitude - minvalue: 60 - - - filter: Background Check - filter variables: [{name: salinity}] - threshold: 5.0 - absolute threshold: 5.0 - - #------------------------------------------------------------------------------- - ### Spike test - #----------------------------------------------------------------------------- - - filter: Spike and Step Check - filter variables: - - name: ObsValue/salinity - dependent: ObsValue/salinity # dy/ - independent: MetaData/depth # dx - count spikes: true - count steps: true - tolerance: - nominal value: 1.0 # PSU nominal, in the case of salinity (not really) - threshold: 0.6 # weird salinity thing - factors: [1,1,0.2] # multiply tolerance, for ranges bounded by... - x boundaries: [0,200,600] # ...these values of x (depth in m) - boundary layer: - x range: [0.0,300.0] # when bounded by these x values (depth in m)... - maximum x interval: [50.0,100.0] # ...and ignore level if dx greater than this - action: - name: reject - - #### Count spikes - #----------------------------------------------------------------------------- - - filter: Variable Assignment # create derived obs value containing only S spikes - assignments: - - name: DerivedMetaData/salinity_spikes - type: int - function: - name: IntObsFunction/ProfileLevelCount - options: - where: - - variable: - name: DiagnosticFlags/spike/salinity - value: is_true - #### Count steps - #----------------------------------------------------------------------------- - - filter: Variable Assignment # create derived obs value containing only S steps - assignments: - - name: DerivedMetaData/salinity_steps - type: int - function: - name: IntObsFunction/ProfileLevelCount - options: - where: - - variable: - name: DiagnosticFlags/step/salinity - value: is_true - #### Count total rejections - #----------------------------------------------------------------------------- - - filter: Variable Assignment # compute sum 2*spikes+steps - assignments: - - name: DerivedMetaData/salinity_rejections - type: int - function: - name: IntObsFunction/LinearCombination - options: - variables: [DerivedMetaData/salinity_spikes, DerivedMetaData/salinity_steps] - coefs: [2,1] - #### Reject entire profile if total rejctions > threshold - #----------------------------------------------------------------------------- - - filter: Perform Action # reject whole profile if 2*spikes+steps>=rejection threshold - where: - - variable: - name: DerivedMetaData/salinity_rejections - minvalue: 3 - action: - name: reject - #------------------------------------------------------------------------------- - ### End of Spike test - #----------------------------------------------------------------------------- - - #----------------------------------------------------------------------------- - ## Filters on the whole profile: - #----------------------------------------------------------------------------- - #------------------------------------------------------------------------------- - ### Ocean Vertical stability test - #----------------------------------------------------------------------------- - - #### get pressure from depth - #----------------------------------------------------------------------------- - - filter: Variable Transforms - Transform: OceanDepthToPressure - ocean depth variable: depth - ocean depth group: MetaData - #### Create diagonostic flags for spike step - #----------------------------------------------------------------------------- - - filter: Create Diagnostic Flags - filter variables: - - name: DerivedObsValue/waterPressure - flags: - - name: DensitySpike - initial value: false - - name: DensityStep - initial value: false - - name: Superadiabat - initial value: false - #### - #----------------------------------------------------------------------------- - - filter: Ocean Vertical Stability Check - where: - - variable: - name: ObsValue/waterTemperature - value: is_valid - filter variables: - - name: DerivedObsValue/waterPressure # density spikes/steps --> flag d - variables: - temperature: ObsValue/waterTemperature - salinity: ObsValue/salinity - pressure: DerivedObsValue/waterPressure - count spikes: true - count steps: true - nominal tolerance: -0.05 - threshold: 0.25 - actions: - - name: set - flag: Superadiabat - - name: reject - #### where there are any density inversions, reject temperature only: - #----------------------------------------------------------------------------- - - filter: Perform Action - filter variables: - - name: ObsValue/waterTemperature - where: - - variable: - name: DiagnosticFlags/Superadiabat/waterPressure - value: is_true - action: - name: reject - #### where density spikes, reject all vars (temperature and salinity): - #----------------------------------------------------------------------------- - - filter: Perform Action - filter variables: - - name: ObsValue/waterTemperature - - name: ObsValue/salinity - where: - - variable: - name: DiagnosticFlags/DensitySpike/waterPressure - value: is_true - action: - name: reject - #### create derived metadata counting levels: - #----------------------------------------------------------------------------- - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/number_of_levels - type: int - function: - name: IntObsFunction/ProfileLevelCount - options: - where: - - variable: - name: ObsValue/waterTemperature - value: is_valid - #### create derived metadata counting spikes and steps: - #----------------------------------------------------------------------------- - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/ocean_density_inversions - type: int - function: - name: IntObsFunction/ProfileLevelCount - options: - where: - - variable: - name: DiagnosticFlags/DensitySpike/waterPressure - value: is_true - - variable: - name: DiagnosticFlags/DensityStep/waterPressure - value: is_true - where operator: or - - #### whole profile is rejected if spikes+steps >= numlev/4, so compute - #### 4*( sum spikes+steps ) minus numlev - #### in order to check it against 0: - #----------------------------------------------------------------------------- - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/ocean_density_rejections - type: int - function: - name: IntObsFunction/LinearCombination - options: - variables: [DerivedMetaData/ocean_density_inversions, DerivedMetaData/number_of_levels] - coefs: [4, -1] - #### reject whole profile if spikes+steps >= numlev/4 AND >= 2: - #----------------------------------------------------------------------------- - - filter: Perform Action - filter variables: - - name: ObsValue/waterTemperature - - name: ObsValue/salinity - where: - - variable: - name: DerivedMetaData/ocean_density_rejections - minvalue: 0 - - variable: - name: DerivedMetaData/ocean_density_inversions - minvalue: 2 - where operator: and - action: - name: reject - #------------------------------------------------------------------------------- - ### End of ocean vertical stability test - #----------------------------------------------------------------------------- - - #----------------------------------------------------------------------------- - ### If T was bad, remove S as well regardless - #----------------------------------------------------------------------------- - - filter: RejectList - where: - - variable: QCflagsData/waterTemperature - minvalue: 1 - defer to post: true diff --git a/parm/jcb-gdas/test/client_integration/gdas-marine-templates.yaml b/parm/jcb-gdas/test/client_integration/gdas-marine-templates.yaml index 2fe3e797a..f258cc92a 100644 --- a/parm/jcb-gdas/test/client_integration/gdas-marine-templates.yaml +++ b/parm/jcb-gdas/test/client_integration/gdas-marine-templates.yaml @@ -36,6 +36,8 @@ number_of_outer_loops: 1 # Model things # ------------ marine_soca_files_path: DATA/soca +marine_vt_levels: 25 +marine_stddev_time: '2024-01-31T21:00:00Z' # Background marine_background_time: '2024-01-31T21:00:00Z' From ea815746fbbff115f5819fe03246e94f5aa2bba2 Mon Sep 17 00:00:00 2001 From: DavidNew-NOAA <134300700+DavidNew-NOAA@users.noreply.github.com> Date: Fri, 1 Nov 2024 10:18:59 -0400 Subject: [PATCH 060/199] Create JCB templates for marine B-matrix (#36) This PR is a companion to Global Workflow PR [#2992](https://github.com/NOAA-EMC/global-workflow/pull/2992) and GDASApp PR [#1346](https://github.com/NOAA-EMC/GDASApp/pull/1346). It adds JCB templates for the input YAMLS of JEDI applications used in the marine B-matrix task in the Global Workflow. --------- Co-authored-by: Dan Holdaway <27729500+danholdaway@users.noreply.github.com> Co-authored-by: Guillaume Vernieres --- .../algorithm/marine/soca_diagb.yaml.j2 | 37 +++++++ .../algorithm/marine/soca_ensb.yaml.j2 | 98 +++++++++++++++++++ .../algorithm/marine/soca_ensweights.yaml.j2 | 37 +++++++ .../algorithm/marine/soca_gridgen.yaml.j2 | 5 + .../soca_parameters_diffusion_hz.yaml.j2 | 37 +++++++ .../soca_parameters_diffusion_vt.yaml.j2 | 34 +++++++ .../marine/soca_setcorscales.yaml.j2 | 23 +++++ .../algorithm/marine/soca_vtscales.yaml.j2 | 13 +++ 8 files changed, 284 insertions(+) create mode 100644 parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 create mode 100644 parm/jcb-gdas/algorithm/marine/soca_ensb.yaml.j2 create mode 100644 parm/jcb-gdas/algorithm/marine/soca_ensweights.yaml.j2 create mode 100644 parm/jcb-gdas/algorithm/marine/soca_gridgen.yaml.j2 create mode 100644 parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_hz.yaml.j2 create mode 100644 parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_vt.yaml.j2 create mode 100644 parm/jcb-gdas/algorithm/marine/soca_setcorscales.yaml.j2 create mode 100644 parm/jcb-gdas/algorithm/marine/soca_vtscales.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 new file mode 100644 index 000000000..7d8e0d6b0 --- /dev/null +++ b/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 @@ -0,0 +1,37 @@ +geometry: + mom6_input_nml: mom_input.nml + fields metadata: ./fields_metadata.yaml + +date: '{{ marine_window_end_iso }}' + +background: + date: '{{ marine_window_end_iso }}' + basename: ./bkg/ + ocn_filename: 'ocean.bkg.f009.nc' + ice_filename: 'ice.bkg.f009.nc' + read_from_file: 1 + +background error: + datadir: ./staticb/ + date: '{{ marine_window_middle_iso }}' + exp: bkgerr_stddev + type: incr + +variables: + name: [tocn, socn, uocn, vocn, hocn, ssh, cicen, hicen, hsnon, mom6_mld] + +rescale: 2.0 # rescales the filtered std. dev. by "rescale" +min sst: 0.0 # Added to sst bkg. err. +max ssh: 0.0 # Limits the amplitude of the unbalanced bkg err +min depth: 500.0 # zero out the bkg. error. at less than min depth +number of halo points: 4 +number of neighbors: 16 + +simple smoothing: + horizontal iterations: 10 + vertical iterations: 1 + +# TODO(G): Too slow for the below scale +#diffusion: +# horizontal: 500.0e3 +# vertical: 3.0 diff --git a/parm/jcb-gdas/algorithm/marine/soca_ensb.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_ensb.yaml.j2 new file mode 100644 index 000000000..01c9777f0 --- /dev/null +++ b/parm/jcb-gdas/algorithm/marine/soca_ensb.yaml.j2 @@ -0,0 +1,98 @@ +# Configuration for the recentering and re-balancing of the ensemble members +geometry: + mom6_input_nml: mom_input.nml + fields metadata: ./fields_metadata.yaml + +date: '{{ marine_window_begin_iso }}' + +layers variable: [hocn] + +increment variables: [tocn, socn, uocn, vocn, ssh, hocn, cicen, hicen, hsnon] + +set increment variables to zero: [ssh] + +vertical geometry: + date: '{{ marine_window_begin_iso }}' + basename: ./INPUT/ + ocn_filename: MOM.res.nc + read_from_file: 3 + +add recentering increment: false + +soca increments: # Could also be states, but they are read as increments + number of increments: '{{ marine_number_ensemble_members }}' + pattern: '%mem%' + template: + date: '{{ marine_window_begin_iso }}' + basename: '{{ enspert_relpath }}/ens/' + ocn_filename: 'ocean.%mem%.nc' + ice_filename: 'ice.%mem%.nc' + read_from_file: 3 + +steric height: + linear variable changes: + - linear variable change name: BalanceSOCA # Only the steric balance is applied + +ensemble mean output: + datadir: ./staticb/ + date: '{{ marine_window_begin_iso }}' + exp: ens_mean + type: incr + +ssh output: + unbalanced: + datadir: ./staticb/ + date: '{{ marine_window_begin_iso }}' + exp: ssh_unbal_stddev + type: incr + + steric: + datadir: ./staticb/ + date: '{{ marine_window_begin_iso }}' + exp: ssh_steric_stddev + type: incr + + total: + datadir: ./staticb/ + date: '{{ marine_window_begin_iso }}' + exp: ssh_total_stddev + type: incr + + explained variance: + datadir: ./staticb/ + date: '{{ marine_window_begin_iso }}' + exp: steric_explained_variance + type: incr + + recentering error: + datadir: ./staticb/ + date: '{{ marine_window_begin_iso }}' + exp: ssh_recentering_error + type: incr + +background error output: + datadir: ./staticb/ + date: '{{ marine_window_begin_iso }}' + exp: bkgerr_stddev + type: incr + +linear variable change: + linear variable changes: + - linear variable change name: BalanceSOCA + +trajectory: + state variables: [tocn, socn, uocn, vocn, ssh, hocn, layer_depth, mld, cicen, hicen, hsnon] + date: '{{ marine_window_begin_iso }}' + basename: ./INPUT/ + ocn_filename: MOM.res.nc + ice_filename: cice.res.nc + read_from_file: 1 + +output increment: + # TODO: Revert this when fms can take more than 128 charactres file names + datadir: '{{ enspert_relpath }}/enspert/' + date: '{{ marine_window_begin_iso }}' + exp: trash + type: incr + output file: 'ocn.pert.steric.%mem%.nc' + pattern: '%mem%' diff --git a/parm/jcb-gdas/algorithm/marine/soca_ensweights.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_ensweights.yaml.j2 new file mode 100644 index 000000000..792bbdcb1 --- /dev/null +++ b/parm/jcb-gdas/algorithm/marine/soca_ensweights.yaml.j2 @@ -0,0 +1,37 @@ +geometry: + mom6_input_nml: mom_input.nml + fields metadata: ./fields_metadata.yaml + +date: '{{ marine_window_middle_iso }}' + +variables: + ice: [cicen, hicen, hsnon] + ocean: [tocn, socn, uocn, vocn, ssh] + +background: + date: '{{ marine_window_begin_iso }}' + basename: ./INPUT/ + ocn_filename: MOM.res.nc + ice_filename: cice.res.nc + read_from_file: 1 + +weights: + # Need to provide weights^2 when reading from file + ice: 0.0025 # 5% of original variance + ocean: 0.0625 # 25% " " + # Apply localized weights to the ocean ens. B + ocean local weights: + - lon: -172.0 + lat: 11.0 + amplitude: -1.0 + length scale: 700.0 + - lon: -160.0 + lat: 12.0 + amplitude: -1.0 + length scale: 700.0 + +output: + datadir: ./ + date: '{{ marine_window_middle_iso }}' + exp: ens_weights + type: incr diff --git a/parm/jcb-gdas/algorithm/marine/soca_gridgen.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_gridgen.yaml.j2 new file mode 100644 index 000000000..34fbdeca6 --- /dev/null +++ b/parm/jcb-gdas/algorithm/marine/soca_gridgen.yaml.j2 @@ -0,0 +1,5 @@ +geometry: + geom_grid_file: soca_gridspec.nc + mom6_input_nml: mom_input.nml + fields metadata: fields_metadata.yaml + rossby file: rossrad.nc diff --git a/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_hz.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_hz.yaml.j2 new file mode 100644 index 000000000..d313e4644 --- /dev/null +++ b/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_hz.yaml.j2 @@ -0,0 +1,37 @@ +geometry: &geom + mom6_input_nml: mom_input.nml + fields metadata: ./fields_metadata.yaml + +background: + read_from_file: 1 + basename: ./INPUT/ + ocn_filename: MOM.res.nc + ice_filename: cice.res.nc + date: '{{ marine_window_end_iso }}' + state variables: [ssh] + +background error: + covariance model: SABER + saber central block: + saber block name: diffusion + geometry: *geom + calibration: + normalization: + method: randomization + iterations: 10000 + + groups: + - horizontal: + model file: + date: '{{ marine_window_end_iso }}' + basename: ./ + ocn_filename: ocn.cor_rh.incr.0001-01-01T00:00:00Z.nc + model variable: ssh + write: + filepath: ./staticb/hz_ocean + + - horizontal: + as gaussian: true + fixed value: 50000.0 + write: + filepath: ./staticb/hz_ice diff --git a/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_vt.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_vt.yaml.j2 new file mode 100644 index 000000000..07f56deab --- /dev/null +++ b/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_vt.yaml.j2 @@ -0,0 +1,34 @@ +geometry: &geom + mom6_input_nml: mom_input.nml + fields metadata: ./fields_metadata.yaml + +background: + read_from_file: 1 + basename: ./INPUT/ + ocn_filename: MOM.res.nc + ice_filename: cice.res.nc + date: '{{ marine_window_middle_iso }}' + state variables: [tocn] + +background error: + covariance model: SABER + saber central block: + saber block name: diffusion + geometry: *geom + calibration: + normalization: + # NOTE, not actually used here, since the normalization spec is only used for hz + method: randomization #< other option is "brute force" + iterations: 1000 #< in the real world you'll want to use 1e4 or so + + groups: + - vertical: + as gaussian: true + levels: '{{ marine_vt_levels }}' + model file: + date: '{{ marine_window_middle_iso }}' + basename: ./ + ocn_filename: vt_scales.nc + model variable: tocn + write: + filepath: ./staticb/vt_ocean diff --git a/parm/jcb-gdas/algorithm/marine/soca_setcorscales.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_setcorscales.yaml.j2 new file mode 100644 index 000000000..b7b80a7f3 --- /dev/null +++ b/parm/jcb-gdas/algorithm/marine/soca_setcorscales.yaml.j2 @@ -0,0 +1,23 @@ +resolution: + mom6_input_nml: mom_input.nml + fields metadata: ./fields_metadata.yaml + +date: "0001-01-01T00:00:00Z" + +corr variables: [ssh] + +scales: + vert layers: 5 # in units of layer + ssh: + rossby mult: 1.00 + min grid mult: 2.0 + +rh output: + datadir: ./ + exp: cor_rh + type: incr + +rv output: + datadir: ./ + exp: cor_rv + type: incr diff --git a/parm/jcb-gdas/algorithm/marine/soca_vtscales.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_vtscales.yaml.j2 new file mode 100644 index 000000000..797af37c3 --- /dev/null +++ b/parm/jcb-gdas/algorithm/marine/soca_vtscales.yaml.j2 @@ -0,0 +1,13 @@ +gridspec_filename: soca_gridspec.nc +restart_filename: ./INPUT/MOM.res.nc +mld_filename: './staticb/ocn.bkgerr_stddev.incr.{{ marine_window_end_iso }}.nc' +output_filename: ./vt_scales.nc +output_variable_vt: Temp +output_variable_hz: ave_ssh + +VT_MIN: 5 +VT_MAX: 15 + +HZ_ROSSBY_MULT: 1.0 +HZ_MAX: 200e3 +HZ_MIN_GRID_MULT: 2.0 From 3ed900a6b26c8c5d00a72abcf21db81199327eb3 Mon Sep 17 00:00:00 2001 From: Azadeh Gholoubi <51101867+azadeh-gh@users.noreply.github.com> Date: Wed, 6 Nov 2024 15:05:34 -0500 Subject: [PATCH 061/199] Add YAML configuration files for SEVIRI M08 and M11 with Thinning and Quality Control (#40) **Description:** This PR adds YAML configuration files for `seviri_m08` and `seviri_m11` into JEDI, along with thinning and observation quality control. The the bufr2ioda python converter for these observations has also been added to the GDASApp repository. **Related Issue:** [Evaluation of SEVIRI YAMLs #756](https://github.com/NOAA-EMC/GDASApp/issues/765) **New files include:** - `observation_chronicle/atmosphere/seviri_m08.yaml` - `observation_chronicle/atmosphere/seviri_m11.yaml` - `observations/atmosphere/seviri_m08.yaml.j2` - `observations/atmosphere/seviri_m11.yaml.j2` --------- Co-authored-by: Azadeh Gholoubi Co-authored-by: Azadeh Gholoubi --- .../atmosphere/seviri_m08.yaml | 25 + .../atmosphere/seviri_m11.yaml | 25 + .../atmosphere/seviri_m08.yaml.j2 | 502 ++++++++++++++++++ .../atmosphere/seviri_m11.yaml.j2 | 502 ++++++++++++++++++ 4 files changed, 1054 insertions(+) create mode 100644 parm/jcb-gdas/observation_chronicle/atmosphere/seviri_m08.yaml create mode 100644 parm/jcb-gdas/observation_chronicle/atmosphere/seviri_m11.yaml create mode 100644 parm/jcb-gdas/observations/atmosphere/seviri_m08.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/seviri_m11.yaml.j2 diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/seviri_m08.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/seviri_m08.yaml new file mode 100644 index 000000000..5d7ffc59c --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/atmosphere/seviri_m08.yaml @@ -0,0 +1,25 @@ +# Instrument metadata +# ------------------- +commissioned: 2012-05-22T00:00:00 +observer_type: satellite # Type of chronicle to use + +# Instrument initial configuration +# -------------------------------- +channel_variables: + simulated: min + active: min + biascorrtd: min + error: max + error_cld: max + ermax: max + +channel_values: +# sim act bcd err errcld ermax + 4: [ 1, -1, -1, 1.80, 0.00, 2.00 ] + 5: [ 1, 1, -1, 2.50, 0.00, 4.00 ] + 6: [ 1, 1, -1, 2.25, 0.00, 3.50 ] + 7: [ 1, -1, -1, 1.25, 0.00, 2.00 ] + 8: [ 1, -1, -1, 1.25, 0.00, 2.00 ] + 9: [ 1, -1, -1, 1.25, 0.00, 2.00 ] + 10: [ 1, -1, -1, 1.45, 0.00, 2.00 ] + 11: [ 1, -1, -1, 1.25, 0.00, 3.00 ] diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/seviri_m11.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/seviri_m11.yaml new file mode 100644 index 000000000..6ca54025e --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/atmosphere/seviri_m11.yaml @@ -0,0 +1,25 @@ +# Instrument metadata +# ------------------- +commissioned: 2019-06-12T00:00:00 +observer_type: satellite # Type of chronicle to use + +# Instrument initial configuration +# -------------------------------- +channel_variables: + simulated: min + active: min + biascorrtd: min + error: max + error_cld: max + ermax: max + +channel_values: +# sim act bcd err errcld ermax + 4: [ 1, -1, -1, 0.75, 0.00, 2.00 ] + 5: [ 1, 1, -1, 2.50, 0.00, 4.00 ] + 6: [ 1, 1, -1, 2.25, 0.00, 3.50 ] + 7: [ 1, -1, -1, 1.25, 0.00, 2.00 ] + 8: [ 1, -1, -1, 1.25, 0.00, 2.00 ] + 9: [ 1, -1, -1, 0.75, 0.00, 2.00 ] + 10: [ 1, -1, -1, 0.80, 0.00, 2.00 ] + 11: [ 1, -1, -1, 1.25, 0.00, 3.00 ] diff --git a/parm/jcb-gdas/observations/atmosphere/seviri_m08.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/seviri_m08.yaml.j2 new file mode 100644 index 000000000..29936f758 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/seviri_m08.yaml.j2 @@ -0,0 +1,502 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: seviri_m08 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [brightnessTemperature] + channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} + + # Observation Operator + # -------------------- + obs operator: + name: CRTM + Absorbers: [H2O, O3, CO2] + Clouds: [Water, Ice] + Cloud_Fraction: 1.0 + obs options: + Sensor_ID: &{{observation_from_jcb}}_sensor_id seviri_m08 + EndianType: little_endian + CoefficientPath: "{{crtm_coefficient_path}}" + linear obs operator: + Absorbers: [H2O, O3] + + # Observation Bias Correction (VarBC) + # ----------------------------------- + obs bias: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" + variational bc: + predictors: + - name: constant + - name: lapseRate + order: 2 + tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" + - name: lapseRate + tlapse: *{{observation_from_jcb}}_tlapse + - name: emissivityJacobian + - name: sensorScanAngle + var_name: sensorScanPosition + order: 4 + - name: sensorScanAngle + var_name: sensorScanPosition + order: 3 + - name: sensorScanAngle + var_name: sensorScanPosition + order: 2 + - name: sensorScanAngle + var_name: sensorScanPosition + covariance: + minimal required obs number: 20 + variance range: [1.0e-6, 10.0] + step size: 1.0e-4 + largest analysis variance: 10000.0 + prior: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiascovin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovin_suffix}}" + inflation: + ratio: 1.1 + ratio for small dataset: 2.0 + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" + + # Observation Filters (QC) + # ------------------------ + obs prior filters: + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: assign error + error parameter vector: &{{observation_from_jcb}}_err [{{ get_satellite_variable(observation_from_jcb, "error") }}] + + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/surfaceFlag + type: int + function: + name: IntObsFunction/Conditional + options: + defaultvalue: 4 + firstmatchingcase: true + cases: + - where: + - variable: + name: GeoVaLs/water_area_fraction + minvalue: 0.99 + value: 0 + - where: + - variable: + name: GeoVaLs/land_area_fraction + minvalue: 0.99 + value: 1 + - where: + - variable: + name: GeoVaLs/ice_area_fraction + minvalue: 0.99 + value: 2 + - where: + - variable: + name: GeoVaLs/surface_snow_area_fraction + minvalue: 0.99 + value: 3 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/surfaceParam + type: int + function: + name: IntObsFunction/Conditional + options: + defaultvalue: 0 + firstmatchingcase: true + cases: + - where: + - variable: + name: GeoVaLs/water_area_fraction + minvalue: 0.99 + value: 30 + - where: + - variable: + name: GeoVaLs/land_area_fraction + minvalue: 0.99 + value: 15 + - where: + - variable: + name: GeoVaLs/ice_area_fraction + minvalue: 0.99 + value: 20 + - where: + - variable: + name: GeoVaLs/surface_snow_area_fraction + minvalue: 0.99 + value: 15 + + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/thinningCriteria + type: int + function: + name: IntObsFunction/Arithmetic + options: + variables: + - name: DerivedMetaData/surfaceParam + coefs: [1] + + obs post filters: + #Step1: Satellite Zenith Angle Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: MetaData/sensorZenithAngle + maxvalue: 65. + action: + name: reject + +#Step2:Clear-sky fraction check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: MetaData/cloudAmount + maxvalue: 30. + action: + name: reject + +#Step3: Scene homogeneous check + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: ClearSkyStdDev/brightnessTemperature + channels: 5 + is_defined: + where: + - variable: + name: ClearSkyStdDev/brightnessTemperature + channels: 6 + is_defined: + where operator: and + + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: ClearSkyStdDev/brightnessTemperature + channels: 5 + maxvalue: 1.3 + - variable: + name: ClearSkyStdDev/brightnessTemperature + channels: 6 + maxvalue: 1.3 + where operator: and + + +#Step4: Thinning + - filter: Gaussian Thinning + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + horizontal_mesh: 145 + use_reduced_horizontal_grid: true + distance_norm: maximum + time_mesh: 'PT06H' + time_min: '2021-07-31T21:00:00Z' + time_max: '2021-08-01T03:00:00Z' + priority_variable: DerivedMetaData/thinningCriteria + action: + name: reject + +#Step5: Surface type check +# Reject channels 5-6 over land-dominant area + - filter: RejectList + filter variables: + - name: brightnessTemperature + channels: 4, 7-11 + where: + - variable: + name: GeoVaLs/land_area_fraction + minvalue: 0.99 + + - filter: RejectList + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: GeoVaLs/surface_snow_area_fraction + minvalue: 0.99 + + - filter: RejectList + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: GeoVaLs/ice_area_fraction + minvalue: 0.99 + + - filter: RejectList + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: GeoVaLs/land_area_fraction + maxvalue: 0.99 + max_exclusive: true + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + max_exclusive: true + - variable: + name: GeoVaLs/ice_area_fraction + maxvalue: 0.99 + max_exclusive: true + - variable: + name: GeoVaLs/surface_snow_area_fraction + maxvalue: 0.99 + max_exclusive: true + +#Step6: Terrain Check: Do not use when height > 1km + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: GeoVaLs/surface_geopotential_height + maxvalue: 1000.0 + +#Step7: BT range check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + minvalue: 0.0 + maxvalue: 1000.0 + action: + name: reject + +#Step 8: Error Inflation based on topography + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ObsErrorFactorTopo + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsFunction/ObsErrorFactorTopoRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: seviri_m08 + channels: *{{observation_from_jcb}}_simulated_channels + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: DerivedMetaData/ObsErrorFactorTopo + channels: *{{observation_from_jcb}}_simulated_channels + +#Step9: Error Inflation based on TOA transmittance + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorTransmitTopRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + +#Step10: Cloud detection + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/CloudDetectMinResidualIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: &{{observation_from_jcb}}_active_channels [{{ get_satellite_variable(observation_from_jcb, "active") }}] + use_flag_clddet: [-2, -2, -2, -2, -2, -2, -2, -2, -2, -2] + obserr_dtempf: [0.5, 2.0, 4.0, 2.0, 4.0] + error parameter vector: *{{observation_from_jcb}}_err + maxvalue: 1.0e-12 + action: + name: reject + +#Step11: Scene Consistency Check based on ch9 + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: 4, 6-11 + where: + - variable: + name: ClearSkyStdDev/brightnessTemperature + channels: 9 + maxvalue: 0.5 + max_exclusive: true + +#Step12: Gross check + # Reject channels 4, 6-11 if omf > 2 + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: 4, 6-11 + absolute threshold: 2.0 + action: + name: reject + +#Step 13: Error inflation for channels 3-4 based on scene consistency from channel 5 + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: 6-7 + where: + - variable: + name: ClearSkyStdDev/brightnessTemperature_5 + maxvalue: 0.5 + minvalue: 0.4 + min_exclusive: true + action: + name: inflate error + inflation factor: 1.14891 + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: 6-7 + where: + - variable: + name: ClearSkyStdDev/brightnessTemperature_5 + maxvalue: 0.6 + max_exclusive: true + minvalue: 0.5 + min_exclusive: true + action: + name: inflate error + inflation factor: 1.29228 + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: 6-7 + where: + - variable: + name: ClearSkyStdDev/brightnessTemperature_5 + maxvalue: 0.7 + max_exclusive: true + minvalue: 0.6 + action: + name: inflate error + inflation factor: 1.49666 + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: 6-7 + where: + - variable: + name: ClearSkyStdDev/brightnessTemperature_5 + minvalue: 0.7 + action: + name: inflate error + inflation factor: 1.51987 + +#Step14: Near SST Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/NearSSTRetCheckIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: *{{observation_from_jcb}}_active_channels + maxvalue: 1.0e-12 + action: + name: reject + +#Step15: Surface Jacobian + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorSurfJacobianRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + sensor: *{{observation_from_jcb}}_sensor_id + obserr_demisf: [0.01, 0.02, 0.03, 0.02, 0.03] + obserr_dtempf: [0.5, 2.0, 4.0, 2.0, 4.0] + +#Step16: Channels 4, 6-11 Cloud fraction >2 + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: 4, 6-11 + where: + - variable: + name: MetaData/cloudAmount + maxvalue: 2 + +#Step17: Final gross check + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + function absolute threshold: + - name: ObsFunction/ObsErrorBoundIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_latitude: + name: ObsFunction/ObsErrorFactorLatRad + options: + latitude_parameters: [0.0, 0.0, 0.0, 0.0] + obserr_bound_transmittop: + name: ObsFunction/ObsErrorFactorTransmitTopRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_max: [{{ get_satellite_variable(observation_from_jcb, "ermax") }}] + error parameter vector: *{{observation_from_jcb}}_err + action: + name: reject + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 + diff --git a/parm/jcb-gdas/observations/atmosphere/seviri_m11.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/seviri_m11.yaml.j2 new file mode 100644 index 000000000..3b49a02e6 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/seviri_m11.yaml.j2 @@ -0,0 +1,502 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: seviri_m11 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [brightnessTemperature] + channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} + + # Observation Operator + # -------------------- + obs operator: + name: CRTM + Absorbers: [H2O, O3, CO2] + Clouds: [Water, Ice] + Cloud_Fraction: 1.0 + obs options: + Sensor_ID: &{{observation_from_jcb}}_sensor_id seviri_m11 + EndianType: little_endian + CoefficientPath: "{{crtm_coefficient_path}}" + linear obs operator: + Absorbers: [H2O, O3] + + # Observation Bias Correction (VarBC) + # ----------------------------------- + obs bias: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" + variational bc: + predictors: + - name: constant + - name: lapseRate + order: 2 + tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" + - name: lapseRate + tlapse: *{{observation_from_jcb}}_tlapse + - name: emissivityJacobian + - name: sensorScanAngle + var_name: sensorScanPosition + order: 4 + - name: sensorScanAngle + var_name: sensorScanPosition + order: 3 + - name: sensorScanAngle + var_name: sensorScanPosition + order: 2 + - name: sensorScanAngle + var_name: sensorScanPosition + covariance: + minimal required obs number: 20 + variance range: [1.0e-6, 10.0] + step size: 1.0e-4 + largest analysis variance: 10000.0 + prior: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiascovin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovin_suffix}}" + inflation: + ratio: 1.1 + ratio for small dataset: 2.0 + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" + + # Observation Filters (QC) + # ------------------------ + obs prior filters: + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: assign error + error parameter vector: &{{observation_from_jcb}}_err [{{ get_satellite_variable(observation_from_jcb, "error") }}] + + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/surfaceFlag + type: int + function: + name: IntObsFunction/Conditional + options: + defaultvalue: 4 + firstmatchingcase: true + cases: + - where: + - variable: + name: GeoVaLs/water_area_fraction + minvalue: 0.99 + value: 0 + - where: + - variable: + name: GeoVaLs/land_area_fraction + minvalue: 0.99 + value: 1 + - where: + - variable: + name: GeoVaLs/ice_area_fraction + minvalue: 0.99 + value: 2 + - where: + - variable: + name: GeoVaLs/surface_snow_area_fraction + minvalue: 0.99 + value: 3 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/surfaceParam + type: int + function: + name: IntObsFunction/Conditional + options: + defaultvalue: 0 + firstmatchingcase: true + cases: + - where: + - variable: + name: GeoVaLs/water_area_fraction + minvalue: 0.99 + value: 30 + - where: + - variable: + name: GeoVaLs/land_area_fraction + minvalue: 0.99 + value: 15 + - where: + - variable: + name: GeoVaLs/ice_area_fraction + minvalue: 0.99 + value: 20 + - where: + - variable: + name: GeoVaLs/surface_snow_area_fraction + minvalue: 0.99 + value: 15 + + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/thinningCriteria + type: int + function: + name: IntObsFunction/Arithmetic + options: + variables: + - name: DerivedMetaData/surfaceParam + coefs: [1] + + obs post filters: + #Step1: Satellite Zenith Angle Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: MetaData/sensorZenithAngle + maxvalue: 65. + action: + name: reject + +#Step2:Clear-sky fraction check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: MetaData/cloudAmount + maxvalue: 30. + action: + name: reject + +#Step3: Scene homogeneous check + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: ClearSkyStdDev/brightnessTemperature + channels: 5 + is_defined: + where: + - variable: + name: ClearSkyStdDev/brightnessTemperature + channels: 6 + is_defined: + where operator: and + + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: ClearSkyStdDev/brightnessTemperature + channels: 5 + maxvalue: 1.3 + - variable: + name: ClearSkyStdDev/brightnessTemperature + channels: 6 + maxvalue: 1.3 + where operator: and + + +#Step4: Thinning + - filter: Gaussian Thinning + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + horizontal_mesh: 145 + use_reduced_horizontal_grid: true + distance_norm: maximum + time_mesh: 'PT06H' + time_min: '2021-07-31T21:00:00Z' + time_max: '2021-08-01T03:00:00Z' + priority_variable: DerivedMetaData/thinningCriteria + action: + name: reject + +#Step5: Surface type check +# Reject channels 5-6 over land-dominant area + - filter: RejectList + filter variables: + - name: brightnessTemperature + channels: 4, 7-11 + where: + - variable: + name: GeoVaLs/land_area_fraction + minvalue: 0.99 + + - filter: RejectList + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: GeoVaLs/surface_snow_area_fraction + minvalue: 0.99 + + - filter: RejectList + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: GeoVaLs/ice_area_fraction + minvalue: 0.99 + + - filter: RejectList + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: GeoVaLs/land_area_fraction + maxvalue: 0.99 + max_exclusive: true + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + max_exclusive: true + - variable: + name: GeoVaLs/ice_area_fraction + maxvalue: 0.99 + max_exclusive: true + - variable: + name: GeoVaLs/surface_snow_area_fraction + maxvalue: 0.99 + max_exclusive: true + +#Step6: Terrain Check: Do not use when height > 1km + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: GeoVaLs/surface_geopotential_height + maxvalue: 1000.0 + +#Step7: BT range check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + minvalue: 0.0 + maxvalue: 1000.0 + action: + name: reject + +#Step 8: Error Inflation based on topography + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ObsErrorFactorTopo + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsFunction/ObsErrorFactorTopoRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: seviri_m11 + channels: *{{observation_from_jcb}}_simulated_channels + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: DerivedMetaData/ObsErrorFactorTopo + channels: *{{observation_from_jcb}}_simulated_channels + +#Step9: Error Inflation based on TOA transmittance + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorTransmitTopRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + +#Step10: Cloud detection + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/CloudDetectMinResidualIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: &{{observation_from_jcb}}_active_channels [{{ get_satellite_variable(observation_from_jcb, "active") }}] + use_flag_clddet: [-2, -2, -2, -2, -2, -2, -2, -2, -2, -2] + obserr_dtempf: [0.5, 2.0, 4.0, 2.0, 4.0] + error parameter vector: *{{observation_from_jcb}}_err + maxvalue: 1.0e-12 + action: + name: reject + +#Step11: Scene Consistency Check based on ch9 + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: 4, 6-11 + where: + - variable: + name: ClearSkyStdDev/brightnessTemperature + channels: 9 + maxvalue: 0.5 + max_exclusive: true + +#Step12: Gross check + # Reject channels 4, 6-11 if omf > 2 + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: 4, 6-11 + absolute threshold: 2.0 + action: + name: reject + +#Step 13: Error inflation for channels 3-4 based on scene consistency from channel 5 + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: 6-7 + where: + - variable: + name: ClearSkyStdDev/brightnessTemperature_5 + maxvalue: 0.5 + minvalue: 0.4 + min_exclusive: true + action: + name: inflate error + inflation factor: 1.14891 + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: 6-7 + where: + - variable: + name: ClearSkyStdDev/brightnessTemperature_5 + maxvalue: 0.6 + max_exclusive: true + minvalue: 0.5 + min_exclusive: true + action: + name: inflate error + inflation factor: 1.29228 + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: 6-7 + where: + - variable: + name: ClearSkyStdDev/brightnessTemperature_5 + maxvalue: 0.7 + max_exclusive: true + minvalue: 0.6 + action: + name: inflate error + inflation factor: 1.49666 + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: 6-7 + where: + - variable: + name: ClearSkyStdDev/brightnessTemperature_5 + minvalue: 0.7 + action: + name: inflate error + inflation factor: 1.51987 + +#Step14: Near SST Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/NearSSTRetCheckIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: *{{observation_from_jcb}}_active_channels + maxvalue: 1.0e-12 + action: + name: reject + +#Step15: Surface Jacobian + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorSurfJacobianRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + sensor: *{{observation_from_jcb}}_sensor_id + obserr_demisf: [0.01, 0.02, 0.03, 0.02, 0.03] + obserr_dtempf: [0.5, 2.0, 4.0, 2.0, 4.0] + +#Step16: Channels 4, 6-11 Cloud fraction >2 + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: 4, 6-11 + where: + - variable: + name: MetaData/cloudAmount + maxvalue: 2 + +#Step17: Final gross check + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + function absolute threshold: + - name: ObsFunction/ObsErrorBoundIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_latitude: + name: ObsFunction/ObsErrorFactorLatRad + options: + latitude_parameters: [0.0, 0.0, 0.0, 0.0] + obserr_bound_transmittop: + name: ObsFunction/ObsErrorFactorTransmitTopRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_max: [{{ get_satellite_variable(observation_from_jcb, "ermax") }}] + error parameter vector: *{{observation_from_jcb}}_err + action: + name: reject + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 + From 2ae0f844b9aa1f1ba5e9782b177bf737817229e9 Mon Sep 17 00:00:00 2001 From: DavidNew-NOAA <134300700+DavidNew-NOAA@users.noreply.github.com> Date: Fri, 8 Nov 2024 08:55:20 -0500 Subject: [PATCH 062/199] Add more CI to GW-CI (#42) This PR is a companion to [#1365](https://github.com/NOAA-EMC/GDASApp/pull/1365). It adds test reference blocks to the YAMLs for the JEDI-to-FV3 increment converters. --- .../atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 | 18 ++++++++++++++++++ .../fv3jedi_fv3inc_variational.yaml.j2 | 18 ++++++++++++++++++ .../gdas-atmosphere-templates.yaml | 4 ++++ 3 files changed, 40 insertions(+) diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 index 4e6406088..bb53866af 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 @@ -62,3 +62,21 @@ members from template: pattern: '%mem%' nmembers: {{ atmosphere_number_ensemble_members }} zero padding: 3 + +# Optionally test the application +{% if do_testing %} +test: + reference filename: {{test_reference_filename}} +{% if test_output_filename is defined %} + test output filename: {{test_output_filename}} +{% endif %} +{% if log_output_filename is defined %} + log output filename: {{log_output_filename}} +{% endif %} +{% if mpi_pattern is defined %} + mpi pattern: {{mpi_pattern}} +{% endif %} + float relative tolerance: {{test_float_relative_tolerance | default(1.0e-6, true)}} + float absolute tolerance: {{test_float_absolute_tolerance | default(0.0, true) }} + integer tolerance: {{test_integer_tolerance | default(0, true) }} +{% endif %} diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 index 80a36349a..f0d06a6f9 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 @@ -58,3 +58,21 @@ members: filetype: auxgrid gridtype: gaussian filename: ./anl/atminc. + +# Optionally test the application +{% if do_testing %} +test: + reference filename: {{test_reference_filename}} +{% if test_output_filename is defined %} + test output filename: {{test_output_filename}} +{% endif %} +{% if log_output_filename is defined %} + log output filename: {{log_output_filename}} +{% endif %} +{% if mpi_pattern is defined %} + mpi pattern: {{mpi_pattern}} +{% endif %} + float relative tolerance: {{test_float_relative_tolerance | default(1.0e-6, true)}} + float absolute tolerance: {{test_float_absolute_tolerance | default(0.0, true) }} + integer tolerance: {{test_integer_tolerance | default(0, true) }} +{% endif %} diff --git a/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml b/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml index bea2b27fb..4f66e4dfb 100644 --- a/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml +++ b/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml @@ -39,6 +39,10 @@ final_diagnostics_departures: anlmob analysis_variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] number_of_outer_loops: 2 +# Testing +# ------- + +do_testing: false # Model things # ------------ From 62ef2a5c8acf65dcce0afd3f479f24b9451bec40 Mon Sep 17 00:00:00 2001 From: JohnSteffen-NOAA <77806417+JohnSteffen-NOAA@users.noreply.github.com> Date: Tue, 12 Nov 2024 16:49:51 -0500 Subject: [PATCH 063/199] Update to jcb-gdas marine obs yamls (#41) This PR updates the marine observations yamls introduced in [https://github.com/NOAA-EMC/jcb-gdas/commit/3835807a8e229049aef44f1fd21f60bed3584712](url) by modifying the observation errors and additional filters to be consistent with the marine candidate experiment. It addresses this GDASApp issue [https://github.com/NOAA-EMC/GDASApp/issues/1316](url). Additionally, yamls for new sea ice retrievals and SMAP/SMOS retrievals are included. Close https://github.com/NOAA-EMC/GDASApp/issues/1316 Co-authored-by: Guillaume Vernieres --- .../observations/marine/adt_rads_all.yaml.j2 | 14 +- .../marine/icec_abi_g16_l2.yaml.j2 | 45 +++ .../marine/icec_amsu_mb_l2.yaml.j2 | 45 +++ .../marine/icec_atms_n20_l2.yaml.j2 | 45 +++ .../marine/icec_atms_n21_l2.yaml.j2 | 45 +++ .../marine/icec_atms_npp_l2.yaml.j2 | 45 +++ .../marine/icec_gmi_gpm_l2.yaml.j2 | 45 +++ .../marine/icec_ssmis_f17_l2.yaml.j2 | 45 +++ .../marine/icec_viirs_n20_l2_north.yaml.j2 | 45 +++ .../marine/icec_viirs_n20_l2_south.yaml.j2 | 45 +++ .../marine/insitu_profile_argo.yaml.j2 | 279 +++++++++++++++++- .../marine/insitu_profile_bathy.yaml.j2 | 13 +- .../marine/insitu_profile_dbuoy.yaml.j2 | 14 +- .../marine/insitu_profile_dbuoyb.yaml.j2 | 13 +- .../marine/insitu_profile_glider.yaml.j2 | 13 +- .../insitu_profile_marinemammal.yaml.j2 | 13 +- .../marine/insitu_profile_mbuoy.yaml.j2 | 13 +- .../marine/insitu_profile_mbuoyb.yaml.j2 | 13 +- .../marine/insitu_profile_tesac.yaml.j2 | 13 +- .../insitu_profile_tesac_salinity.yaml.j2 | 13 +- .../marine/insitu_profile_xbtctd.yaml.j2 | 13 +- .../marine/insitu_surface_altkob.yaml.j2 | 13 +- .../marine/insitu_surface_trkob.yaml.j2 | 13 +- .../insitu_surface_trkob_salinity.yaml.j2 | 13 +- .../observations/marine/sss_smap_l2.yaml.j2 | 43 +++ .../observations/marine/sss_smos_l2.yaml.j2 | 43 +++ .../marine/sst_abi_g16_l3c.yaml.j2 | 16 +- .../marine/sst_abi_g17_l3c.yaml.j2 | 16 +- .../marine/sst_ahi_h08_l3c.yaml.j2 | 16 +- .../marine/sst_avhrr_ma_l3u.yaml.j2 | 10 +- .../marine/sst_avhrr_mb_l3u.yaml.j2 | 10 +- .../marine/sst_avhrr_mc_l3u.yaml.j2 | 10 +- .../marine/sst_viirs_n20_l3u.yaml.j2 | 10 +- .../marine/sst_viirs_npp_l3u.yaml.j2 | 10 +- 34 files changed, 996 insertions(+), 56 deletions(-) create mode 100644 parm/jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/icec_atms_n20_l2.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/icec_atms_n21_l2.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/icec_atms_npp_l2.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_north.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_south.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/sss_smap_l2.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/sss_smos_l2.yaml.j2 diff --git a/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 b/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 index da532ecaf..e3be83cfb 100644 --- a/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 @@ -40,8 +40,8 @@ options: variables: [GeoVaLs/mesoscale_representation_error, ObsError/absoluteDynamicTopography] - coefs: [0.1, - 1.0] + coefs: [0.2, + 2.0] - filter: Domain Check where: - variable: { name: GeoVaLs/sea_ice_area_fraction} @@ -50,3 +50,13 @@ where: - variable: {name: GeoVaLs/distance_from_coast} minvalue: 100e3 + - filter: BlackList + where: + - variable: + name: MetaData/latitude + minvalue: 4 + maxvalue: 30 + - variable: + name: MetaData/longitude + minvalue: 32 + maxvalue: 56 diff --git a/parm/jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 new file mode 100644 index 000000000..c63389837 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 @@ -0,0 +1,45 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaIceFraction] + io pool: + max pool size: 1 + get values: + time interpolation: linear + obs operator: + name: Identity + observation alias file: obsop_name_map.yaml + obs error: + covariance model: diagonal + obs filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + minvalue: 0.9 + - filter: Bounds Check + minvalue: 0.0 + maxvalue: 1.0 + #- filter: Background Check + # threshold: 5.0 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_surface_temperature} + maxvalue: 2.0 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_surface_temperature} + maxvalue: 0.0 + action: + name: inflate error + inflation factor: 2.0 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/distance_from_coast} + minvalue: 100e3 diff --git a/parm/jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 new file mode 100644 index 000000000..c63389837 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 @@ -0,0 +1,45 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaIceFraction] + io pool: + max pool size: 1 + get values: + time interpolation: linear + obs operator: + name: Identity + observation alias file: obsop_name_map.yaml + obs error: + covariance model: diagonal + obs filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + minvalue: 0.9 + - filter: Bounds Check + minvalue: 0.0 + maxvalue: 1.0 + #- filter: Background Check + # threshold: 5.0 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_surface_temperature} + maxvalue: 2.0 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_surface_temperature} + maxvalue: 0.0 + action: + name: inflate error + inflation factor: 2.0 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/distance_from_coast} + minvalue: 100e3 diff --git a/parm/jcb-gdas/observations/marine/icec_atms_n20_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_atms_n20_l2.yaml.j2 new file mode 100644 index 000000000..c63389837 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/icec_atms_n20_l2.yaml.j2 @@ -0,0 +1,45 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaIceFraction] + io pool: + max pool size: 1 + get values: + time interpolation: linear + obs operator: + name: Identity + observation alias file: obsop_name_map.yaml + obs error: + covariance model: diagonal + obs filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + minvalue: 0.9 + - filter: Bounds Check + minvalue: 0.0 + maxvalue: 1.0 + #- filter: Background Check + # threshold: 5.0 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_surface_temperature} + maxvalue: 2.0 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_surface_temperature} + maxvalue: 0.0 + action: + name: inflate error + inflation factor: 2.0 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/distance_from_coast} + minvalue: 100e3 diff --git a/parm/jcb-gdas/observations/marine/icec_atms_n21_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_atms_n21_l2.yaml.j2 new file mode 100644 index 000000000..c63389837 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/icec_atms_n21_l2.yaml.j2 @@ -0,0 +1,45 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaIceFraction] + io pool: + max pool size: 1 + get values: + time interpolation: linear + obs operator: + name: Identity + observation alias file: obsop_name_map.yaml + obs error: + covariance model: diagonal + obs filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + minvalue: 0.9 + - filter: Bounds Check + minvalue: 0.0 + maxvalue: 1.0 + #- filter: Background Check + # threshold: 5.0 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_surface_temperature} + maxvalue: 2.0 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_surface_temperature} + maxvalue: 0.0 + action: + name: inflate error + inflation factor: 2.0 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/distance_from_coast} + minvalue: 100e3 diff --git a/parm/jcb-gdas/observations/marine/icec_atms_npp_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_atms_npp_l2.yaml.j2 new file mode 100644 index 000000000..c63389837 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/icec_atms_npp_l2.yaml.j2 @@ -0,0 +1,45 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaIceFraction] + io pool: + max pool size: 1 + get values: + time interpolation: linear + obs operator: + name: Identity + observation alias file: obsop_name_map.yaml + obs error: + covariance model: diagonal + obs filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + minvalue: 0.9 + - filter: Bounds Check + minvalue: 0.0 + maxvalue: 1.0 + #- filter: Background Check + # threshold: 5.0 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_surface_temperature} + maxvalue: 2.0 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_surface_temperature} + maxvalue: 0.0 + action: + name: inflate error + inflation factor: 2.0 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/distance_from_coast} + minvalue: 100e3 diff --git a/parm/jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 new file mode 100644 index 000000000..c63389837 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 @@ -0,0 +1,45 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaIceFraction] + io pool: + max pool size: 1 + get values: + time interpolation: linear + obs operator: + name: Identity + observation alias file: obsop_name_map.yaml + obs error: + covariance model: diagonal + obs filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + minvalue: 0.9 + - filter: Bounds Check + minvalue: 0.0 + maxvalue: 1.0 + #- filter: Background Check + # threshold: 5.0 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_surface_temperature} + maxvalue: 2.0 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_surface_temperature} + maxvalue: 0.0 + action: + name: inflate error + inflation factor: 2.0 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/distance_from_coast} + minvalue: 100e3 diff --git a/parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 new file mode 100644 index 000000000..c63389837 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 @@ -0,0 +1,45 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaIceFraction] + io pool: + max pool size: 1 + get values: + time interpolation: linear + obs operator: + name: Identity + observation alias file: obsop_name_map.yaml + obs error: + covariance model: diagonal + obs filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + minvalue: 0.9 + - filter: Bounds Check + minvalue: 0.0 + maxvalue: 1.0 + #- filter: Background Check + # threshold: 5.0 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_surface_temperature} + maxvalue: 2.0 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_surface_temperature} + maxvalue: 0.0 + action: + name: inflate error + inflation factor: 2.0 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/distance_from_coast} + minvalue: 100e3 diff --git a/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_north.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_north.yaml.j2 new file mode 100644 index 000000000..c63389837 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_north.yaml.j2 @@ -0,0 +1,45 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaIceFraction] + io pool: + max pool size: 1 + get values: + time interpolation: linear + obs operator: + name: Identity + observation alias file: obsop_name_map.yaml + obs error: + covariance model: diagonal + obs filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + minvalue: 0.9 + - filter: Bounds Check + minvalue: 0.0 + maxvalue: 1.0 + #- filter: Background Check + # threshold: 5.0 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_surface_temperature} + maxvalue: 2.0 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_surface_temperature} + maxvalue: 0.0 + action: + name: inflate error + inflation factor: 2.0 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/distance_from_coast} + minvalue: 100e3 diff --git a/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_south.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_south.yaml.j2 new file mode 100644 index 000000000..c63389837 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_south.yaml.j2 @@ -0,0 +1,45 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaIceFraction] + io pool: + max pool size: 1 + get values: + time interpolation: linear + obs operator: + name: Identity + observation alias file: obsop_name_map.yaml + obs error: + covariance model: diagonal + obs filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + minvalue: 0.9 + - filter: Bounds Check + minvalue: 0.0 + maxvalue: 1.0 + #- filter: Background Check + # threshold: 5.0 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_surface_temperature} + maxvalue: 2.0 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_surface_temperature} + maxvalue: 0.0 + action: + name: inflate error + inflation factor: 2.0 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/distance_from_coast} + minvalue: 100e3 diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_argo.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_argo.yaml.j2 index 0d121a013..12b09fa5d 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_argo.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_argo.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: insitu_profile_argo + name: {{observation_from_jcb}} obsdatain: engine: type: H5File @@ -59,3 +59,280 @@ minvalue: -2.5 maxvalue: 40.0 + #----------------------------------------------------------------------------- + ### Regional range tests + #----------------------------------------------------------------------------- + + #### Red Sea + #----------------------------------------------------------------------------- + #### + #### the document linked here describes Red sea as the are between 10N, 40E; + #### 20N, 50E; 30N, 30E; 10N, 40E. But that would also include Gulf of Aden. + #### A more reasonable choice seemed to be a box that extends from 10 N to + #### 30 N and 30 E to 45 East . + + - filter: Bounds Check + filter variables: [{name: waterTemperature}] + minvalue: 21.7 + maxvalue: 40.0 + where: + - variable: + name: MetaData/latitude + minvalue: 10 + maxvalue: 30 + - variable: + name: MetaData/longitude + minvalue: 30 + maxvalue: 45 + + #### Mediterranean Sea + #----------------------------------------------------------------------------- + ##### Area 1/3 for Mediterranean Sea + - filter: Bounds Check + filter variables: [{name: waterTemperature}] + minvalue: 10.0 + maxvalue: 40.0 + where: + - variable: + name: MetaData/latitude + minvalue: 30 + maxvalue: 40 + - variable: + name: MetaData/longitude + minvalue: -6 + maxvalue: 40 + ##### Area 2/3 for Mediterranean Sea + - filter: Bounds Check + filter variables: [{name: waterTemperature}] + minvalue: 10.0 + maxvalue: 40.0 + where: + - variable: + name: MetaData/latitude + minvalue: 40 + maxvalue: 41.5 + - variable: + name: MetaData/longitude + minvalue: 20 + maxvalue: 30 + ##### Area 3/3 for Mediterranean Sea + - filter: Bounds Check + filter variables: [{name: waterTemperature}] + minvalue: 10.0 + maxvalue: 40.0 + where: + - variable: + name: MetaData/latitude + minvalue: 40 + maxvalue: 46 + - variable: + name: MetaData/longitude + minvalue: 0 + maxvalue: 20 + + #### Northwestern shelves + #----------------------------------------------------------------------------- + - filter: Bounds Check + filter variables: [{name: waterTemperature}] + minvalue: -2.0 + maxvalue: 24.0 + where: + - variable: + name: MetaData/latitude + minvalue: 50 + maxvalue: 60 + - variable: + name: MetaData/longitude + minvalue: -20 + maxvalue: 10 + #### Southwestern shelves + #----------------------------------------------------------------------------- + - filter: Bounds Check + filter variables: [{name: waterTemperature}] + minvalue: -2.0 + maxvalue: 30 + where: + - variable: + name: MetaData/latitude + minvalue: 25 + maxvalue: 50 + - variable: + name: MetaData/longitude + minvalue: -30 + maxvalue: 0 + + #### Arctic Ocean + #----------------------------------------------------------------------------- + - filter: Bounds Check + filter variables: [{name: waterTemperature}] + minvalue: -1.92 + maxvalue: 25.0 + where: + - variable: + name: MetaData/latitude + minvalue: 60 + + - filter: Background Check + filter variables: [{name: waterTemperature}] + threshold: 5.0 + absolute threshold: 5.0 + + ## Filters for S: + #------------------------------------------------------------------------------- + #----------------------------------------------------------------------------- + ### Global range test + #----------------------------------------------------------------------------- + - filter: Bounds Check + filter variables: [{name: salinity}] + minvalue: 2.0 + maxvalue: 41.0 + + #----------------------------------------------------------------------------- + ### Regional range test + #----------------------------------------------------------------------------- + #### Red Sea + #----------------------------------------------------------------------------- + #### + #### the document linked here describes Red sea as the are between 10N, 40E; + #### 20N, 50E; 30N, 30E; 10N, 40E. But that would also include Gulf of Aden. + #### A more reasonable choice seemed to be a box that extends from 10 N to + #### 30 N and 30 E to 45 East . + + - filter: Bounds Check + filter variables: [{name: salinity}] + minvalue: 2.0 + maxvalue: 41.0 + where: + - variable: + name: MetaData/latitude + minvalue: 10 + maxvalue: 30 + - variable: + name: MetaData/longitude + minvalue: 30 + maxvalue: 45 + + #### Mediterranean Sea + #----------------------------------------------------------------------------- + ##### Area 1/3 for Mediterranean Sea + - filter: Bounds Check + filter variables: [{name: salinity}] + minvalue: 2.0 + maxvalue: 40.0 + where: + - variable: + name: MetaData/latitude + minvalue: 30 + maxvalue: 40 + - variable: + name: MetaData/longitude + minvalue: -6 + maxvalue: 40 + ##### Area 2/3 for Mediterranean Sea + - filter: Bounds Check + filter variables: [{name: salinity}] + minvalue: 2.0 + maxvalue: 40.0 + where: + - variable: + name: MetaData/latitude + minvalue: 40 + maxvalue: 41.5 + - variable: + name: MetaData/longitude + minvalue: 20 + maxvalue: 30 + ##### Area 3/3 for Mediterranean Sea + - filter: Bounds Check + filter variables: [{name: salinity}] + minvalue: 2.0 + maxvalue: 40.0 + where: + - variable: + name: MetaData/latitude + minvalue: 40 + maxvalue: 46 + - variable: + name: MetaData/longitude + minvalue: 0 + maxvalue: 20 + + + #### Northwestern shelves + #----------------------------------------------------------------------------- + - filter: Bounds Check + filter variables: [{name: salinity}] + minvalue: 0.0 + maxvalue: 37.0 + where: + - variable: + name: MetaData/latitude + minvalue: 50 + maxvalue: 60 + - variable: + name: MetaData/longitude + minvalue: -20 + maxvalue: 10 + + #### Southwestern shelves + #----------------------------------------------------------------------------- + - filter: Bounds Check + filter variables: [{name: salinity}] + minvalue: 0.0 + maxvalue: 38 + where: + - variable: + name: MetaData/latitude + minvalue: 25 + maxvalue: 50 + - variable: + name: MetaData/longitude + minvalue: -30 + maxvalue: 0 + + #### Arctic Ocean + #----------------------------------------------------------------------------- + - filter: Bounds Check + filter variables: [{name: salinity}] + minvalue: 2.0 + maxvalue: 40.0 + where: + - variable: + name: MetaData/latitude + minvalue: 60 + + - filter: Background Check + filter variables: [{name: salinity}] + threshold: 5.0 + absolute threshold: 5.0 + + #----------------------------------------------------------------------------- + - filter: RejectList + where: + - variable: QCflagsData/waterTemperature + minvalue: 1 + defer to post: true + + #-------------------------------------------------------------------------- + ### Set in situ obs multiplier coef for T,S + #------------------------------------------------------------------------ + - filter: Perform Action + action: + name: assign error + error function: + name: ObsFunction/LinearCombination + options: + variables: + - ObsError/waterTemperature + coefs: + - 1000.0 + - filter: Perform Action + action: + name: assign error + error function: + name: ObsFunction/LinearCombination + options: + variables: + - ObsError/salinity + coefs: + - 1000.0 diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 index 8223f3617..e379c3b35 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: insitu_profile_bathy + name: {{observation_from_jcb}} obsdatain: engine: type: H5File @@ -15,3 +15,14 @@ name: InsituTemperature obs error: covariance model: diagonal + obs filters: + - filter: Perform Action + action: + name: assign error + error function: + name: ObsFunction/LinearCombination + options: + variables: + - ObsError/waterTemperature + coefs: + - 1000.0 diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_dbuoy.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_dbuoy.yaml.j2 index cec36c123..e8a9ca3bc 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_dbuoy.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_dbuoy.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: insitu_profile_dbuoy + name: {{observation_from_jcb}} obsdatain: engine: type: H5File @@ -20,3 +20,15 @@ - name: waterTemperature obs error: covariance model: diagonal + obs filters: + - filter: Perform Action + action: + name: assign error + error function: + name: ObsFunction/LinearCombination + options: + variables: + - ObsError/waterTemperature + coefs: + - 1000.0 + diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_dbuoyb.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_dbuoyb.yaml.j2 index 5cec54d67..8cdc4e166 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_dbuoyb.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_dbuoyb.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: insitu_profile_dbuoyb + name: {{observation_from_jcb}} obsdatain: engine: type: H5File @@ -20,3 +20,14 @@ - name: waterTemperature obs error: covariance model: diagonal + obs filters: + - filter: Perform Action + action: + name: assign error + error function: + name: ObsFunction/LinearCombination + options: + variables: + - ObsError/waterTemperature + coefs: + - 1000.0 diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_glider.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_glider.yaml.j2 index 1c9bab393..8cdc4e166 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_glider.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_glider.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: insitu_profile_glider + name: {{observation_from_jcb}} obsdatain: engine: type: H5File @@ -20,3 +20,14 @@ - name: waterTemperature obs error: covariance model: diagonal + obs filters: + - filter: Perform Action + action: + name: assign error + error function: + name: ObsFunction/LinearCombination + options: + variables: + - ObsError/waterTemperature + coefs: + - 1000.0 diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_marinemammal.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_marinemammal.yaml.j2 index ad4456565..8cdc4e166 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_marinemammal.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_marinemammal.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: insitu_profile_marinemammal + name: {{observation_from_jcb}} obsdatain: engine: type: H5File @@ -20,3 +20,14 @@ - name: waterTemperature obs error: covariance model: diagonal + obs filters: + - filter: Perform Action + action: + name: assign error + error function: + name: ObsFunction/LinearCombination + options: + variables: + - ObsError/waterTemperature + coefs: + - 1000.0 diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_mbuoy.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_mbuoy.yaml.j2 index bf783ca76..8cdc4e166 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_mbuoy.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_mbuoy.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: insitu_profile_mbuoy + name: {{observation_from_jcb}} obsdatain: engine: type: H5File @@ -20,3 +20,14 @@ - name: waterTemperature obs error: covariance model: diagonal + obs filters: + - filter: Perform Action + action: + name: assign error + error function: + name: ObsFunction/LinearCombination + options: + variables: + - ObsError/waterTemperature + coefs: + - 1000.0 diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_mbuoyb.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_mbuoyb.yaml.j2 index 78c261734..8cdc4e166 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_mbuoyb.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_mbuoyb.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: insitu_profile_mbuoyb + name: {{observation_from_jcb}} obsdatain: engine: type: H5File @@ -20,3 +20,14 @@ - name: waterTemperature obs error: covariance model: diagonal + obs filters: + - filter: Perform Action + action: + name: assign error + error function: + name: ObsFunction/LinearCombination + options: + variables: + - ObsError/waterTemperature + coefs: + - 1000.0 diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_tesac.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_tesac.yaml.j2 index 0c9af5751..d0986b9a4 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_tesac.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_tesac.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: insitu_profile_tesac + name: {{observation_from_jcb}} obsdatain: engine: type: H5File @@ -18,3 +18,14 @@ - name: InsituTemperature variables: - name: waterTemperature + obs filters: + - filter: Perform Action + action: + name: assign error + error function: + name: ObsFunction/LinearCombination + options: + variables: + - ObsError/waterTemperature + coefs: + - 1000.0 diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_tesac_salinity.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_tesac_salinity.yaml.j2 index 36c50035e..4e8b3bc17 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_tesac_salinity.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_tesac_salinity.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: insitu_profile_tesac_salinity + name: {{observation_from_jcb}} obsdatain: engine: type: H5File @@ -20,3 +20,14 @@ interpolation method: linear obs error: covariance model: diagonal + obs filters: + - filter: Perform Action + action: + name: assign error + error function: + name: ObsFunction/LinearCombination + options: + variables: + - ObsError/salinity + coefs: + - 1000.0 diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_xbtctd.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_xbtctd.yaml.j2 index aee91cf79..8cdc4e166 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_xbtctd.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_xbtctd.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: insitu_profile_xbtctd + name: {{observation_from_jcb}} obsdatain: engine: type: H5File @@ -20,3 +20,14 @@ - name: waterTemperature obs error: covariance model: diagonal + obs filters: + - filter: Perform Action + action: + name: assign error + error function: + name: ObsFunction/LinearCombination + options: + variables: + - ObsError/waterTemperature + coefs: + - 1000.0 diff --git a/parm/jcb-gdas/observations/marine/insitu_surface_altkob.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_surface_altkob.yaml.j2 index 063c3ec51..cf7da6427 100644 --- a/parm/jcb-gdas/observations/marine/insitu_surface_altkob.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_surface_altkob.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: insitu_surface_altkob + name: {{observation_from_jcb}} obsdatain: engine: type: H5File @@ -16,3 +16,14 @@ observation alias file: obsop_name_map.yaml obs error: covariance model: diagonal + obs filters: + - filter: Perform Action + action: + name: assign error + error function: + name: ObsFunction/LinearCombination + options: + variables: + - ObsError/seaSurfaceTemperature + coefs: + - 1000.0 diff --git a/parm/jcb-gdas/observations/marine/insitu_surface_trkob.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_surface_trkob.yaml.j2 index 625129273..833fdac9c 100644 --- a/parm/jcb-gdas/observations/marine/insitu_surface_trkob.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_surface_trkob.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: insitu_surface_trkob + name: {{observation_from_jcb}} obsdatain: engine: type: H5File @@ -16,3 +16,14 @@ observation alias file: ./obsop_name_map.yaml obs error: covariance model: diagonal + obs filters: + - filter: Perform Action + action: + name: assign error + error function: + name: ObsFunction/LinearCombination + options: + variables: + - ObsError/seaSurfaceTemperature + coefs: + - 1000.0 diff --git a/parm/jcb-gdas/observations/marine/insitu_surface_trkob_salinity.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_surface_trkob_salinity.yaml.j2 index 5ba67e942..3bf2085ff 100644 --- a/parm/jcb-gdas/observations/marine/insitu_surface_trkob_salinity.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_surface_trkob_salinity.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: insitu_surface_trkob_salinity + name: {{observation_from_jcb}} obsdatain: engine: type: H5File @@ -16,3 +16,14 @@ observation alias file: ./obsop_name_map.yaml obs error: covariance model: diagonal + obs filters: + - filter: Perform Action + action: + name: assign error + error function: + name: ObsFunction/LinearCombination + options: + variables: + - ObsError/seaSurfaceSalinity + coefs: + - 1000.0 diff --git a/parm/jcb-gdas/observations/marine/sss_smap_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/sss_smap_l2.yaml.j2 new file mode 100644 index 000000000..c8b50b2d2 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/sss_smap_l2.yaml.j2 @@ -0,0 +1,43 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaSurfaceSalinity] + io pool: + max pool size: 1 + get values: + time interpolation: linear + obs operator: + name: Identity + observation alias file: obsop_name_map.yaml + obs error: + covariance model: diagonal + obs filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + minvalue: 0.9 + - filter: Bounds Check + minvalue: 0.1 + maxvalue: 40.0 + - filter: Background Check + threshold: 5.0 + - filter: Domain Check + action: + name: passivate + where: + - variable: {name: GeoVaLs/sea_surface_temperature} + minvalue: 15.0 + ## Gaussian_Thinning is having problems with LETKF, try again later + # - filter: Gaussian_Thinning + # horizontal_mesh: 25.0 #km + - filter: Domain Check + where: + - variable: {name: GeoVaLs/distance_from_coast} + minvalue: 100e3 diff --git a/parm/jcb-gdas/observations/marine/sss_smos_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/sss_smos_l2.yaml.j2 new file mode 100644 index 000000000..c8b50b2d2 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/sss_smos_l2.yaml.j2 @@ -0,0 +1,43 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaSurfaceSalinity] + io pool: + max pool size: 1 + get values: + time interpolation: linear + obs operator: + name: Identity + observation alias file: obsop_name_map.yaml + obs error: + covariance model: diagonal + obs filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + minvalue: 0.9 + - filter: Bounds Check + minvalue: 0.1 + maxvalue: 40.0 + - filter: Background Check + threshold: 5.0 + - filter: Domain Check + action: + name: passivate + where: + - variable: {name: GeoVaLs/sea_surface_temperature} + minvalue: 15.0 + ## Gaussian_Thinning is having problems with LETKF, try again later + # - filter: Gaussian_Thinning + # horizontal_mesh: 25.0 #km + - filter: Domain Check + where: + - variable: {name: GeoVaLs/distance_from_coast} + minvalue: 100e3 diff --git a/parm/jcb-gdas/observations/marine/sst_abi_g16_l3c.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_abi_g16_l3c.yaml.j2 index 184fc29ee..2d912345c 100644 --- a/parm/jcb-gdas/observations/marine/sst_abi_g16_l3c.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sst_abi_g16_l3c.yaml.j2 @@ -19,6 +19,12 @@ obs error: covariance model: diagonal obs filters: + - filter: Domain Check + action: + name: passivate + where: + - variable: {name: ObsValue/seaSurfaceTemperature} + maxvalue: -4.0 - filter: Domain Check where: - variable: {name: GeoVaLs/sea_area_fraction} @@ -40,10 +46,10 @@ where: - variable: {name: GeoVaLs/sea_surface_temperature} minvalue: -1.0 - #- filter: Domain Check - # where: - # - variable: {name: GeoVaLs/distance_from_coast} - # minvalue: 100e3 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/distance_from_coast} + minvalue: 100e3 - filter: Perform Action action: name: assign error @@ -53,4 +59,4 @@ variables: - ObsError/seaSurfaceTemperature coefs: - - 0.05 + - 1.0 diff --git a/parm/jcb-gdas/observations/marine/sst_abi_g17_l3c.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_abi_g17_l3c.yaml.j2 index 184fc29ee..2d912345c 100644 --- a/parm/jcb-gdas/observations/marine/sst_abi_g17_l3c.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sst_abi_g17_l3c.yaml.j2 @@ -19,6 +19,12 @@ obs error: covariance model: diagonal obs filters: + - filter: Domain Check + action: + name: passivate + where: + - variable: {name: ObsValue/seaSurfaceTemperature} + maxvalue: -4.0 - filter: Domain Check where: - variable: {name: GeoVaLs/sea_area_fraction} @@ -40,10 +46,10 @@ where: - variable: {name: GeoVaLs/sea_surface_temperature} minvalue: -1.0 - #- filter: Domain Check - # where: - # - variable: {name: GeoVaLs/distance_from_coast} - # minvalue: 100e3 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/distance_from_coast} + minvalue: 100e3 - filter: Perform Action action: name: assign error @@ -53,4 +59,4 @@ variables: - ObsError/seaSurfaceTemperature coefs: - - 0.05 + - 1.0 diff --git a/parm/jcb-gdas/observations/marine/sst_ahi_h08_l3c.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_ahi_h08_l3c.yaml.j2 index 184fc29ee..2d912345c 100644 --- a/parm/jcb-gdas/observations/marine/sst_ahi_h08_l3c.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sst_ahi_h08_l3c.yaml.j2 @@ -19,6 +19,12 @@ obs error: covariance model: diagonal obs filters: + - filter: Domain Check + action: + name: passivate + where: + - variable: {name: ObsValue/seaSurfaceTemperature} + maxvalue: -4.0 - filter: Domain Check where: - variable: {name: GeoVaLs/sea_area_fraction} @@ -40,10 +46,10 @@ where: - variable: {name: GeoVaLs/sea_surface_temperature} minvalue: -1.0 - #- filter: Domain Check - # where: - # - variable: {name: GeoVaLs/distance_from_coast} - # minvalue: 100e3 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/distance_from_coast} + minvalue: 100e3 - filter: Perform Action action: name: assign error @@ -53,4 +59,4 @@ variables: - ObsError/seaSurfaceTemperature coefs: - - 0.05 + - 1.0 diff --git a/parm/jcb-gdas/observations/marine/sst_avhrr_ma_l3u.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_avhrr_ma_l3u.yaml.j2 index 184fc29ee..63f847a92 100644 --- a/parm/jcb-gdas/observations/marine/sst_avhrr_ma_l3u.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sst_avhrr_ma_l3u.yaml.j2 @@ -40,10 +40,10 @@ where: - variable: {name: GeoVaLs/sea_surface_temperature} minvalue: -1.0 - #- filter: Domain Check - # where: - # - variable: {name: GeoVaLs/distance_from_coast} - # minvalue: 100e3 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/distance_from_coast} + minvalue: 100e3 - filter: Perform Action action: name: assign error @@ -53,4 +53,4 @@ variables: - ObsError/seaSurfaceTemperature coefs: - - 0.05 + - 1.0 diff --git a/parm/jcb-gdas/observations/marine/sst_avhrr_mb_l3u.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_avhrr_mb_l3u.yaml.j2 index 184fc29ee..63f847a92 100644 --- a/parm/jcb-gdas/observations/marine/sst_avhrr_mb_l3u.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sst_avhrr_mb_l3u.yaml.j2 @@ -40,10 +40,10 @@ where: - variable: {name: GeoVaLs/sea_surface_temperature} minvalue: -1.0 - #- filter: Domain Check - # where: - # - variable: {name: GeoVaLs/distance_from_coast} - # minvalue: 100e3 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/distance_from_coast} + minvalue: 100e3 - filter: Perform Action action: name: assign error @@ -53,4 +53,4 @@ variables: - ObsError/seaSurfaceTemperature coefs: - - 0.05 + - 1.0 diff --git a/parm/jcb-gdas/observations/marine/sst_avhrr_mc_l3u.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_avhrr_mc_l3u.yaml.j2 index 184fc29ee..63f847a92 100644 --- a/parm/jcb-gdas/observations/marine/sst_avhrr_mc_l3u.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sst_avhrr_mc_l3u.yaml.j2 @@ -40,10 +40,10 @@ where: - variable: {name: GeoVaLs/sea_surface_temperature} minvalue: -1.0 - #- filter: Domain Check - # where: - # - variable: {name: GeoVaLs/distance_from_coast} - # minvalue: 100e3 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/distance_from_coast} + minvalue: 100e3 - filter: Perform Action action: name: assign error @@ -53,4 +53,4 @@ variables: - ObsError/seaSurfaceTemperature coefs: - - 0.05 + - 1.0 diff --git a/parm/jcb-gdas/observations/marine/sst_viirs_n20_l3u.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_viirs_n20_l3u.yaml.j2 index 184fc29ee..63f847a92 100644 --- a/parm/jcb-gdas/observations/marine/sst_viirs_n20_l3u.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sst_viirs_n20_l3u.yaml.j2 @@ -40,10 +40,10 @@ where: - variable: {name: GeoVaLs/sea_surface_temperature} minvalue: -1.0 - #- filter: Domain Check - # where: - # - variable: {name: GeoVaLs/distance_from_coast} - # minvalue: 100e3 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/distance_from_coast} + minvalue: 100e3 - filter: Perform Action action: name: assign error @@ -53,4 +53,4 @@ variables: - ObsError/seaSurfaceTemperature coefs: - - 0.05 + - 1.0 diff --git a/parm/jcb-gdas/observations/marine/sst_viirs_npp_l3u.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_viirs_npp_l3u.yaml.j2 index 184fc29ee..63f847a92 100644 --- a/parm/jcb-gdas/observations/marine/sst_viirs_npp_l3u.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sst_viirs_npp_l3u.yaml.j2 @@ -40,10 +40,10 @@ where: - variable: {name: GeoVaLs/sea_surface_temperature} minvalue: -1.0 - #- filter: Domain Check - # where: - # - variable: {name: GeoVaLs/distance_from_coast} - # minvalue: 100e3 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/distance_from_coast} + minvalue: 100e3 - filter: Perform Action action: name: assign error @@ -53,4 +53,4 @@ variables: - ObsError/seaSurfaceTemperature coefs: - - 0.05 + - 1.0 From 9961f01e75a20aff7233c1eefbfbac916499182c Mon Sep 17 00:00:00 2001 From: DavidNew-NOAA <134300700+DavidNew-NOAA@users.noreply.github.com> Date: Thu, 14 Nov 2024 14:02:28 -0500 Subject: [PATCH 064/199] Make do_testing an optional parameter (#44) Does the same thing as jcb-algorithms PR [#8](https://github.com/NOAA-EMC/jcb-algorithms/pull/8) --- .../jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 | 2 +- .../algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 index bb53866af..723abc609 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 @@ -64,7 +64,7 @@ members from template: zero padding: 3 # Optionally test the application -{% if do_testing %} +{% if do_testing | default(false) %} test: reference filename: {{test_reference_filename}} {% if test_output_filename is defined %} diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 index f0d06a6f9..b795b0621 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 @@ -60,7 +60,7 @@ members: filename: ./anl/atminc. # Optionally test the application -{% if do_testing %} +{% if do_testing | default(false) %} test: reference filename: {{test_reference_filename}} {% if test_output_filename is defined %} From 1f8af26b6015de001b6a22f5f2e749f9eb95f4ee Mon Sep 17 00:00:00 2001 From: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com> Date: Fri, 15 Nov 2024 14:45:30 -0500 Subject: [PATCH 065/199] Update variables to be consistent with updated jedi hashes (#45) This PR updates variables to be consistent with the updated jedi hashes in GDASApp branch [`feature/resume_nightly`](https://github.com/NOAA-EMC/GDASApp/tree/feature/resume_nightly) Resolves #43 --- .../atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 | 2 +- .../atmosphere/fv3jedi_fv3inc_variational.yaml.j2 | 2 +- .../algorithm/marine/soca_2cice_antarctic.yaml.j2 | 4 ++-- .../algorithm/marine/soca_2cice_arctic.yaml.j2 | 4 ++-- parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 | 2 +- .../marine/soca_parameters_diffusion_hz.yaml.j2 | 4 ++-- .../marine/soca_parameters_diffusion_vt.yaml.j2 | 4 ++-- .../algorithm/marine/soca_setcorscales.yaml.j2 | 4 ++-- parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 | 4 ++-- .../model/atmosphere/atmosphere_background.yaml.j2 | 4 ++-- .../atmosphere_background_ensemble.yaml.j2 | 4 ++-- ...phere_background_error_hybrid_gsibec_bump.yaml.j2 | 12 ++++++------ ...atmosphere_background_error_static_gsibec.yaml.j2 | 4 ++-- parm/jcb-gdas/model/marine/marine_background.yaml.j2 | 2 +- .../marine_background_error_static_diffusion.yaml.j2 | 8 ++++---- .../atmosphere-lgetkf/ascatw.ascat_metop-b.yaml.j2 | 2 +- .../atmosphere-lgetkf/conventional_ps.yaml.j2 | 4 ++-- .../atmosphere-lgetkf/satwnd.abi_goes-16.yaml.j2 | 2 +- .../atmosphere/ascatw.ascat_metop-b.yaml.j2 | 2 +- .../observations/atmosphere/conventional_ps.yaml.j2 | 4 ++-- .../atmosphere/satwnd.abi_goes-16.yaml.j2 | 2 +- .../atmosphere/scatwind_ascat_metop-a.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 | 4 ++-- 23 files changed, 43 insertions(+), 43 deletions(-) diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 index 723abc609..576a133eb 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 @@ -1,6 +1,6 @@ variable change: variable change name: Model2GeoVaLs - input variables: &bkgvars [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr,surface_geopotential_height] + input variables: &bkgvars [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr,hgtsfc] output variables: &fv3incrvars [ua,va,t,sphum,ice_wat,liq_wat,o3mr,delp,delz] jedi increment variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] fv3 increment variables: *fv3incrvars diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 index b795b0621..dd60ebb75 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 @@ -1,6 +1,6 @@ variable change: variable change name: Model2GeoVaLs - input variables: &bkgvars [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr,surface_geopotential_height] + input variables: &bkgvars [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr,hgtsfc] output variables: &fv3incrvars [ua,va,t,sphum,ice_wat,liq_wat,o3mr,delp,delz] jedi increment variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] fv3 increment variables: *fv3incrvars diff --git a/parm/jcb-gdas/algorithm/marine/soca_2cice_antarctic.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_2cice_antarctic.yaml.j2 index 73fa36753..c740c5327 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_2cice_antarctic.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_2cice_antarctic.yaml.j2 @@ -26,7 +26,7 @@ variable change: tstep: PT1H cice output: restart: {{ ice_rst }} - output variables: [tocn, socn, hocn, cicen, hicen, hsnon] + output variables: [sea_water_potential_temperature, sea_water_salinity, sea_water_cell_thickness, sea_ice_area_fraction, sea_ice_thickness, sea_ice_snow_thickness] states: - input: @@ -35,7 +35,7 @@ states: ocn_filename: {{ ocn_ana }} ice_filename: {{ ice_ana }} date: '{{ fcst_begin }}' - state variables: [tocn, socn, hocn, cicen, hicen, hsnon] + state variables: [sea_water_potential_temperature, sea_water_salinity, sea_water_cell_thickness, sea_ice_area_fraction, sea_ice_thickness, sea_ice_snow_thickness] output: datadir: Data exp: soca2cice diff --git a/parm/jcb-gdas/algorithm/marine/soca_2cice_arctic.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_2cice_arctic.yaml.j2 index e686eea08..178b8faf6 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_2cice_arctic.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_2cice_arctic.yaml.j2 @@ -23,7 +23,7 @@ variable change: tstep: PT1H cice output: restart: {{ ice_rst }} - output variables: [tocn, socn, hocn, cicen, hicen, hsnon] + output variables: [sea_water_potential_temperature, sea_water_salinity, sea_water_cell_thickness, sea_ice_area_fraction, sea_ice_thickness, sea_ice_snow_thickness] states: - input: @@ -32,7 +32,7 @@ states: ocn_filename: {{ ocn_ana }} ice_filename: {{ ice_ana }} date: '{{ fcst_begin }}' - state variables: [tocn, socn, hocn, cicen, hicen, hsnon] + state variables: [sea_water_potential_temperature, sea_water_salinity, sea_water_cell_thickness, sea_ice_area_fraction, sea_ice_thickness, sea_ice_snow_thickness] output: datadir: Data exp: soca2cice diff --git a/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 index 7d8e0d6b0..02633aca4 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 @@ -18,7 +18,7 @@ background error: type: incr variables: - name: [tocn, socn, uocn, vocn, hocn, ssh, cicen, hicen, hsnon, mom6_mld] + name: [sea_water_potential_temperature, sea_water_salinity, eastward_sea_water_velocity, northward_sea_water_velocity, sea_water_cell_thickness, sea_surface_height_above_geoid, sea_ice_area_fraction, sea_ice_thickness, sea_ice_snow_thickness, mom6_mld] rescale: 2.0 # rescales the filtered std. dev. by "rescale" min sst: 0.0 # Added to sst bkg. err. diff --git a/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_hz.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_hz.yaml.j2 index d313e4644..c79d469b7 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_hz.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_hz.yaml.j2 @@ -8,7 +8,7 @@ background: ocn_filename: MOM.res.nc ice_filename: cice.res.nc date: '{{ marine_window_end_iso }}' - state variables: [ssh] + state variables: [sea_surface_height_above_geoid] background error: covariance model: SABER @@ -26,7 +26,7 @@ background error: date: '{{ marine_window_end_iso }}' basename: ./ ocn_filename: ocn.cor_rh.incr.0001-01-01T00:00:00Z.nc - model variable: ssh + model variable: sea_surface_height_above_geoid write: filepath: ./staticb/hz_ocean diff --git a/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_vt.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_vt.yaml.j2 index 07f56deab..8e23f2d9b 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_vt.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_vt.yaml.j2 @@ -8,7 +8,7 @@ background: ocn_filename: MOM.res.nc ice_filename: cice.res.nc date: '{{ marine_window_middle_iso }}' - state variables: [tocn] + state variables: [sea_water_potential_temperature] background error: covariance model: SABER @@ -29,6 +29,6 @@ background error: date: '{{ marine_window_middle_iso }}' basename: ./ ocn_filename: vt_scales.nc - model variable: tocn + model variable: sea_water_potential_temperature write: filepath: ./staticb/vt_ocean diff --git a/parm/jcb-gdas/algorithm/marine/soca_setcorscales.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_setcorscales.yaml.j2 index b7b80a7f3..c1a1ec86c 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_setcorscales.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_setcorscales.yaml.j2 @@ -4,11 +4,11 @@ resolution: date: "0001-01-01T00:00:00Z" -corr variables: [ssh] +corr variables: [sea_surface_height_above_geoid] scales: vert layers: 5 # in units of layer - ssh: + sea_surface_height_above_geoid: rossby mult: 1.00 min grid mult: 2.0 diff --git a/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 b/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 index ececb6bea..deaec761a 100644 --- a/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 @@ -4,9 +4,9 @@ geometry: date: '{{ marine_window_begin }}' -layers variable: [hocn] +layers variable: [sea_water_cell_thickness] -increment variables: [tocn, socn, uocn, vocn, ssh] +increment variables: [sea_water_potential_temperature, sea_water_salinity, eastward_sea_water_velocity, northward_sea_water_velocity, sea_surface_height_above_geoid] vertical geometry: date: '{{ marine_window_begin }}' diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 index 580fc68ac..24a45f4b5 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 @@ -5,6 +5,6 @@ ufs soil nlev: 4 ufs fields split by level: [smc,stc] datetime: "{{ atmosphere_background_time_iso }}" filenames: [ {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_atmf006.nc, {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_sfcf006.nc ] -state variables: [ua,va,t,delp,ps,sphum,ice_wat,liq_wat,o3mr,surface_geopotential_height, - slmsk,sheleg,tsea,vtype,stype,vfrac,stc,smc,snwdphMeters, +state variables: [ua,va,t,delp,ps,sphum,ice_wat,liq_wat,o3mr,hgtsfc, + slmsk,sheleg,tmpsfc,vtype,stype,vfrac,stc,smc,snwdphMeters, u_srf,v_srf,f10m] diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_background_ensemble.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background_ensemble.yaml.j2 index 16b980e40..216788579 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_background_ensemble.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background_ensemble.yaml.j2 @@ -6,8 +6,8 @@ members from template: provider: ufs ufs soil nlev: 4 ufs fields split by level: [smc,stc] - state variables: [ua,va,t,delz,delp,ps,sphum,ice_wat,liq_wat,o3mr,surface_geopotential_height, - slmsk,sheleg,tsea,vtype,stype,vfrac,stc,smc,snwdphMeters, + state variables: [ua,va,t,delz,delp,ps,sphum,ice_wat,liq_wat,o3mr,hgtsfc, + slmsk,sheleg,tmpsfc,vtype,stype,vfrac,stc,smc,snwdphMeters, u_srf,v_srf,f10m] datapath: {{ atmosphere_background_ensemble_path }} filename is datetime templated: true diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 index e5b06aa69..588091752 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 @@ -21,8 +21,8 @@ components: debugging mode: false linear variable change: linear variable change name: Control2Analysis - input variables: [eastward_wind,northward_wind,air_temperature,surface_pressure, - specific_humidity,cloud_liquid_ice,cloud_liquid_water, + input variables: [eastward_wind,northward_wind,air_temperature,air_pressure_at_surface, + water_vapor_mixing_ratio_wrt_moist_air,cloud_liquid_ice,cloud_liquid_water, mole_fraction_of_ozone_in_air] output variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] weight: @@ -45,8 +45,8 @@ components: localization method: SABER saber central block: saber block name: BUMP_NICAS - active variables: [eastward_wind,northward_wind,air_temperature,surface_pressure, - specific_humidity,cloud_liquid_ice,cloud_liquid_water, + active variables: [eastward_wind,northward_wind,air_temperature,air_pressure_at_surface, + water_vapor_mixing_ratio_wrt_moist_air,cloud_liquid_ice,cloud_liquid_water, ozone_mass_mixing_ratio ] read: general: @@ -198,8 +198,8 @@ components: value: 0.3 linear variable change: linear variable change name: Control2Analysis - input variables: [eastward_wind,northward_wind,air_temperature,surface_pressure, - specific_humidity,cloud_liquid_ice,cloud_liquid_water, + input variables: [eastward_wind,northward_wind,air_temperature,air_pressure_at_surface, + water_vapor_mixing_ratio_wrt_moist_air,cloud_liquid_ice,cloud_liquid_water, ozone_mass_mixing_ratio ] output variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] weight: diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_gsibec.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_gsibec.yaml.j2 index 242ce4f2e..1133aaf7d 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_gsibec.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_gsibec.yaml.j2 @@ -19,7 +19,7 @@ saber outer blocks: debugging mode: false linear variable change: linear variable change name: Control2Analysis - input variables: [eastward_wind,northward_wind,air_temperature,surface_pressure, - specific_humidity,cloud_liquid_ice,cloud_liquid_water, + input variables: [eastward_wind,northward_wind,air_temperature,air_pressure_at_surface, + water_vapor_mixing_ratio_wrt_moist_air,cloud_liquid_ice,cloud_liquid_water, mole_fraction_of_ozone_in_air] output variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] diff --git a/parm/jcb-gdas/model/marine/marine_background.yaml.j2 b/parm/jcb-gdas/model/marine/marine_background.yaml.j2 index d0047e85e..205f58337 100644 --- a/parm/jcb-gdas/model/marine/marine_background.yaml.j2 +++ b/parm/jcb-gdas/model/marine/marine_background.yaml.j2 @@ -3,4 +3,4 @@ basename: {{ marine_background_path}} ocn_filename: MOM.res.nc ice_filename: cice.res.nc date: "{{ marine_background_time }}" -state variables: [cicen, hicen, hsnon, socn, tocn, uocn, vocn, ssh, hocn, mld, layer_depth] +state variables: [sea_ice_area_fraction, sea_ice_thickness, sea_ice_snow_thickness, sea_water_salinity, sea_water_potential_temperature, eastward_sea_water_velocity, northward_sea_water_velocity, sea_surface_height_above_geoid, sea_water_cell_thickness, ocean_mixed_layer_thickness, sea_water_depth] diff --git a/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 b/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 index bb5de6194..0c57790d6 100644 --- a/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 +++ b/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 @@ -3,13 +3,13 @@ saber central block: saber block name: diffusion read: groups: - - variables: [tocn, socn, ssh] + - variables: [sea_water_potential_temperature, sea_water_salinity, sea_surface_height_above_geoid] horizontal: filepath: ./staticb/hz_ocean vertical: levels: {{marine_vt_levels}} filepath: ./staticb/vt_ocean - - variables: [cicen] + - variables: [sea_ice_area_fraction] horizontal: filepath: ./staticb/hz_ice @@ -24,7 +24,7 @@ saber outer blocks: read_from_file: 3 linear variable change: - input variables: [cicen, hicen, hsnon, socn, tocn, uocn, vocn, ssh] - output variables: [cicen, hicen, hsnon, socn, tocn, uocn, vocn, ssh] + input variables: [sea_ice_area_fraction, sea_ice_thickness, sea_ice_snow_thickness, sea_water_salinity, sea_water_potential_temperature, eastward_sea_water_velocity, northward_sea_water_velocity, sea_surface_height_above_geoid] + output variables: [sea_ice_area_fraction, sea_ice_thickness, sea_ice_snow_thickness, sea_water_salinity, sea_water_potential_temperature, eastward_sea_water_velocity, northward_sea_water_velocity, sea_surface_height_above_geoid] linear variable changes: - linear variable change name: BalanceSOCA diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/ascatw.ascat_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/ascatw.ascat_metop-b.yaml.j2 index 35a8a06b7..831fb80d7 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/ascatw.ascat_metop-b.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/ascatw.ascat_metop-b.yaml.j2 @@ -110,7 +110,7 @@ - variable: ObsType/windEastward is_in: 290 # - variable: GeoVaLs/surface_temperature - - variable: GeoVaLs/surface_temperature_where_land + - variable: GeoVaLs/skin_temperature_at_surface_where_land maxvalue: 273. action: name: reject diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml.j2 index f094ca94e..49f99a5a9 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml.j2 @@ -27,7 +27,7 @@ - name: stationPressure da_psfc_scheme: GSI station_altitude: height - geovar_sfc_geomz: surface_altitude + geovar_sfc_geomz: height_above_mean_sea_level_at_surface geovar_geomz: geopotential_height # Linear Observation Operator @@ -264,7 +264,7 @@ function: name: ObsFunction/ObsErrorFactorSfcPressure options: - geovar_sfc_geomz: surface_altitude + geovar_sfc_geomz: height_above_mean_sea_level_at_surface geovar_geomz: geopotential_height station_altitude: height diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-16.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-16.yaml.j2 index e5543ba0d..d3b214ca8 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-16.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-16.yaml.j2 @@ -448,7 +448,7 @@ - variable: name: ObsType/windEastward is_in: 247 - reference: GeoVaLs/surface_pressure + reference: GeoVaLs/air_pressure_at_surface value: MetaData/pressure maxvalue: -11000. # within 110 hPa above surface pressure, negative p-diff action: diff --git a/parm/jcb-gdas/observations/atmosphere/ascatw.ascat_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ascatw.ascat_metop-b.yaml.j2 index bb3d5de74..88ee802ef 100644 --- a/parm/jcb-gdas/observations/atmosphere/ascatw.ascat_metop-b.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ascatw.ascat_metop-b.yaml.j2 @@ -107,7 +107,7 @@ - variable: ObsType/windEastward is_in: 290 # - variable: GeoVaLs/surface_temperature - - variable: GeoVaLs/surface_temperature_where_land + - variable: GeoVaLs/skin_temperature_at_surface_where_land maxvalue: 273. action: name: reject diff --git a/parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 index d8d90f9c8..1ecd386ab 100644 --- a/parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 @@ -24,7 +24,7 @@ - name: stationPressure da_psfc_scheme: GSI station_altitude: height - geovar_sfc_geomz: surface_altitude + geovar_sfc_geomz: height_above_mean_sea_level_at_surface geovar_geomz: geopotential_height # Linear Observation Operator @@ -261,7 +261,7 @@ function: name: ObsFunction/ObsErrorFactorSfcPressure options: - geovar_sfc_geomz: surface_altitude + geovar_sfc_geomz: height_above_mean_sea_level_at_surface geovar_geomz: geopotential_height station_altitude: height diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 index e01f17ce4..94b0458cc 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 @@ -445,7 +445,7 @@ - variable: name: ObsType/windEastward is_in: 247 - reference: GeoVaLs/surface_pressure + reference: GeoVaLs/air_pressure_at_surface value: MetaData/pressure maxvalue: -11000. # within 110 hPa above surface pressure, negative p-diff action: diff --git a/parm/jcb-gdas/observations/atmosphere/scatwind_ascat_metop-a.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/scatwind_ascat_metop-a.yaml.j2 index 0c1a74514..774fef9f6 100644 --- a/parm/jcb-gdas/observations/atmosphere/scatwind_ascat_metop-a.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/scatwind_ascat_metop-a.yaml.j2 @@ -107,7 +107,7 @@ - variable: ObsType/windEastward is_in: 290 # - variable: GeoVaLs/surface_temperature - - variable: GeoVaLs/surface_temperature_where_land + - variable: GeoVaLs/skin_temperature_at_surface_where_land maxvalue: 273. action: name: reject diff --git a/parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 index aff438b96..224c81459 100644 --- a/parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 @@ -34,7 +34,7 @@ - name: specificHumidity - name: SfcPCorrected da_psfc_scheme: GSI - geovar_sfc_geomz: surface_geopotential_height + geovar_sfc_geomz: geopotential_height_at_surface geovar_geomz: geopotential_height variables: - name: stationPressure @@ -106,7 +106,7 @@ error_min: 100.0 # 1 mb error_max: 300.0 # 3 mb geovar_geomz: geopotential_height - geovar_sfc_geomz: surface_geopotential_height + geovar_sfc_geomz: geopotential_height_at_surface # # Gross error check with (O - B) / ObsError greater than threshold - filter: Background Check From 0cc6ad6c81a67b395c09f3dc5805da2d78090dad Mon Sep 17 00:00:00 2001 From: Anna Shlyaeva Date: Thu, 21 Nov 2024 12:06:39 -0700 Subject: [PATCH 066/199] Add soca->cice global file (#47) --- .../marine/soca_2cice_arctic.yaml.j2 | 40 ------------------- ...ctic.yaml.j2 => soca_2cice_global.yaml.j2} | 25 +++++++----- 2 files changed, 14 insertions(+), 51 deletions(-) delete mode 100644 parm/jcb-gdas/algorithm/marine/soca_2cice_arctic.yaml.j2 rename parm/jcb-gdas/algorithm/marine/{soca_2cice_antarctic.yaml.j2 => soca_2cice_global.yaml.j2} (76%) diff --git a/parm/jcb-gdas/algorithm/marine/soca_2cice_arctic.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_2cice_arctic.yaml.j2 deleted file mode 100644 index 178b8faf6..000000000 --- a/parm/jcb-gdas/algorithm/marine/soca_2cice_arctic.yaml.j2 +++ /dev/null @@ -1,40 +0,0 @@ -input geometry: - mom6_input_nml: mom_input.nml - fields metadata: ./fields_metadata.yaml - -output geometry: - mom6_input_nml: mom_input.nml - fields metadata: ./fields_metadata.yaml - -variable change: - variable change name: Soca2Cice - do inverse: false - seaice edge: 0.8 - shuffle: false - rescale prior: - min hice: 0.5 - min hsno: 0.1 - domain: arctic - cice background state: - restart: {{ ice_rst }} - ncat: 5 - ice_lev: 7 - sno_lev: 1 - tstep: PT1H - cice output: - restart: {{ ice_rst }} - output variables: [sea_water_potential_temperature, sea_water_salinity, sea_water_cell_thickness, sea_ice_area_fraction, sea_ice_thickness, sea_ice_snow_thickness] - -states: -- input: - read_from_file: 1 - basename: ./ - ocn_filename: {{ ocn_ana }} - ice_filename: {{ ice_ana }} - date: '{{ fcst_begin }}' - state variables: [sea_water_potential_temperature, sea_water_salinity, sea_water_cell_thickness, sea_ice_area_fraction, sea_ice_thickness, sea_ice_snow_thickness] - output: - datadir: Data - exp: soca2cice - type: fc - date: '{{ fcst_begin }}' diff --git a/parm/jcb-gdas/algorithm/marine/soca_2cice_antarctic.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_2cice_global.yaml.j2 similarity index 76% rename from parm/jcb-gdas/algorithm/marine/soca_2cice_antarctic.yaml.j2 rename to parm/jcb-gdas/algorithm/marine/soca_2cice_global.yaml.j2 index c740c5327..0798d02e6 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_2cice_antarctic.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_2cice_global.yaml.j2 @@ -1,29 +1,32 @@ input geometry: mom6_input_nml: mom_input.nml fields metadata: fields_metadata.yaml - geom_grid_file: soca_gridspec.nc output geometry: mom6_input_nml: mom_input.nml fields metadata: fields_metadata.yaml - geom_grid_file: soca_gridspec.nc variable change: variable change name: Soca2Cice - do inverse: false - seaice edge: 0.9 - shuffle: false # seg. fault when true - rescale prior: - min hice: 0.5 - min hsno: 0.1 - domain: antarctic + arctic: + seaice edge: 0.8 + shuffle: false + rescale prior: + rescale: true + min hice: 0.5 + min hsno: 0.1 + antarctic: + seaice edge: 0.9 + shuffle: false # seg. fault when true + rescale prior: + rescale: true + min hice: 0.5 + min hsno: 0.1 cice background state: restart: {{ ice_rst }} - grid: ' ' ncat: 5 ice_lev: 7 sno_lev: 1 - tstep: PT1H cice output: restart: {{ ice_rst }} output variables: [sea_water_potential_temperature, sea_water_salinity, sea_water_cell_thickness, sea_ice_area_fraction, sea_ice_thickness, sea_ice_snow_thickness] From 63454f07222bc53ef6cb8f9e464894ecc30a61b1 Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Tue, 26 Nov 2024 14:37:02 -0500 Subject: [PATCH 067/199] more changes for switching aerosol DA to JCB (#48) --- .../aero/aero_gen_bmatrix_diagb.yaml.j2 | 21 ++++++++++++++++++- .../algorithm/aero/aero_obs_staging.yaml.j2 | 8 +++++++ 2 files changed, 28 insertions(+), 1 deletion(-) create mode 100644 parm/jcb-gdas/algorithm/aero/aero_obs_staging.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diagb.yaml.j2 b/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diagb.yaml.j2 index d1f3edac2..d90756eb3 100644 --- a/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diagb.yaml.j2 +++ b/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diagb.yaml.j2 @@ -73,7 +73,26 @@ variables: - mass_fraction_of_sea_salt003_in_air - mass_fraction_of_sea_salt004_in_air -rescale: {{ aero_diagb_rescale_factor }} # rescales the filtered std. dev. by "rescale" +global rescale: + geometry: + fms initialization: + namelist filename: "{{aero_fv3jedi_files_path}}/fmsmpp.nml" + field table filename: "{{aero_fv3jedi_files_path}}/field_table" + akbk: "{{aero_fv3jedi_files_path}}/akbk.nc4" + layout: + - {{ aero_layout_x }} + - {{ aero_layout_y }} + npx: {{ aero_npx_clim_b }} + npy: {{ aero_npy_clim_b }} + npz: {{ aero_npz_clim_b }} + field metadata override: "{{aero_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" + rescale stddev: + filetype: fms restart + skip coupler file: true + datapath: {{ aero_rescale_b_path }} + filename_trcr: rescale.fv_tracer.res.nc + filename_cplr: rescale.coupler.res + number of halo points: {{ aero_diagb_n_halo }} number of neighbors: {{ aero_diagb_n_neighbors }} simple smoothing: diff --git a/parm/jcb-gdas/algorithm/aero/aero_obs_staging.yaml.j2 b/parm/jcb-gdas/algorithm/aero/aero_obs_staging.yaml.j2 new file mode 100644 index 000000000..0e5c9920b --- /dev/null +++ b/parm/jcb-gdas/algorithm/aero/aero_obs_staging.yaml.j2 @@ -0,0 +1,8 @@ +mkdir: + - '{{aero_obsdatain_path}}' +copy: + {% for observation_from_jcb in observations %} + {% if use_observer(observation_from_jcb) %} + - ['{{aero_obsdataroot_path}}/{{aero_obsdatain_prefix}}{{observation_from_jcb}}{{aero_obsdatain_suffix}}', '{{aero_obsdatain_path}}'] + {% endif %} + {% endfor %} From 18a5f099c190f17114d91e46319cf8daf6ad5496 Mon Sep 17 00:00:00 2001 From: Guillaume Vernieres Date: Mon, 2 Dec 2024 13:19:10 -0500 Subject: [PATCH 068/199] Bug fixes and changes necessary to run the Ens. var (#46) --- .../marine/soca_2cice_global.yaml.j2 | 17 +++++++-- .../algorithm/marine/soca_diagb.yaml.j2 | 15 ++++++-- .../algorithm/marine/soca_ensb.yaml.j2 | 31 ++++++++++++---- .../algorithm/marine/soca_ensweights.yaml.j2 | 13 +++++-- .../soca_parameters_diffusion_hz.yaml.j2 | 13 ++++--- ..._background_error_static_diffusion.yaml.j2 | 35 +++++++++++++++---- 6 files changed, 101 insertions(+), 23 deletions(-) diff --git a/parm/jcb-gdas/algorithm/marine/soca_2cice_global.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_2cice_global.yaml.j2 index 0798d02e6..8d13d46b0 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_2cice_global.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_2cice_global.yaml.j2 @@ -29,7 +29,13 @@ variable change: sno_lev: 1 cice output: restart: {{ ice_rst }} - output variables: [sea_water_potential_temperature, sea_water_salinity, sea_water_cell_thickness, sea_ice_area_fraction, sea_ice_thickness, sea_ice_snow_thickness] + output variables: + - sea_water_potential_temperature + - sea_water_salinity + - sea_water_cell_thickness + - sea_ice_area_fraction + - sea_ice_thickness + - sea_ice_snow_thickness states: - input: @@ -38,7 +44,14 @@ states: ocn_filename: {{ ocn_ana }} ice_filename: {{ ice_ana }} date: '{{ fcst_begin }}' - state variables: [sea_water_potential_temperature, sea_water_salinity, sea_water_cell_thickness, sea_ice_area_fraction, sea_ice_thickness, sea_ice_snow_thickness] + state variables: + - sea_water_potential_temperature + - sea_water_salinity + - sea_water_cell_thickness + - sea_ice_area_fraction + - sea_ice_thickness + - sea_ice_snow_thickness + output: datadir: Data exp: soca2cice diff --git a/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 index 02633aca4..30e9c6864 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 @@ -18,9 +18,20 @@ background error: type: incr variables: - name: [sea_water_potential_temperature, sea_water_salinity, eastward_sea_water_velocity, northward_sea_water_velocity, sea_water_cell_thickness, sea_surface_height_above_geoid, sea_ice_area_fraction, sea_ice_thickness, sea_ice_snow_thickness, mom6_mld] + name: + - sea_water_potential_temperature + - sea_water_salinity + - eastward_sea_water_velocity + - northward_sea_water_velocity + - sea_water_cell_thickness + - sea_surface_height_above_geoid + - sea_ice_area_fraction + - sea_ice_thickness + - sea_ice_snow_thickness + - mom6_mld -rescale: 2.0 # rescales the filtered std. dev. by "rescale" +vertical e-folding scale: 500.0 +rescale: 1.0 # rescales the filtered std. dev. by "rescale" min sst: 0.0 # Added to sst bkg. err. max ssh: 0.0 # Limits the amplitude of the unbalanced bkg err min depth: 500.0 # zero out the bkg. error. at less than min depth diff --git a/parm/jcb-gdas/algorithm/marine/soca_ensb.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_ensb.yaml.j2 index 01c9777f0..852850f15 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_ensb.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_ensb.yaml.j2 @@ -5,11 +5,20 @@ geometry: date: '{{ marine_window_begin_iso }}' -layers variable: [hocn] +layers variable: [sea_water_cell_thickness] -increment variables: [tocn, socn, uocn, vocn, ssh, hocn, cicen, hicen, hsnon] +increment variables: +- sea_water_potential_temperature +- sea_water_salinity +- eastward_sea_water_velocity +- northward_sea_water_velocity +- sea_surface_height_above_geoid +- sea_water_cell_thickness +- sea_ice_area_fraction +- sea_ice_thickness +- sea_ice_snow_thickness -set increment variables to zero: [ssh] +set increment variables to zero: [sea_surface_height_above_geoid] vertical geometry: date: '{{ marine_window_begin_iso }}' @@ -81,7 +90,18 @@ linear variable change: - linear variable change name: BalanceSOCA trajectory: - state variables: [tocn, socn, uocn, vocn, ssh, hocn, layer_depth, mld, cicen, hicen, hsnon] + state variables: + - sea_water_potential_temperature + - sea_water_salinity + - eastward_sea_water_velocity + - northward_sea_water_velocity + - sea_surface_height_above_geoid + - sea_water_cell_thickness + - sea_water_depth + - ocean_mixed_layer_thickness + - sea_ice_area_fraction + - sea_ice_thickness + - sea_ice_snow_thickness date: '{{ marine_window_begin_iso }}' basename: ./INPUT/ ocn_filename: MOM.res.nc @@ -89,8 +109,7 @@ trajectory: read_from_file: 1 output increment: - # TODO: Revert this when fms can take more than 128 charactres file names - datadir: '{{ enspert_relpath }}/enspert/' + datadir: '{{ enspert_relpath }}/' date: '{{ marine_window_begin_iso }}' exp: trash type: incr diff --git a/parm/jcb-gdas/algorithm/marine/soca_ensweights.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_ensweights.yaml.j2 index 792bbdcb1..9f818b34f 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_ensweights.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_ensweights.yaml.j2 @@ -5,8 +5,17 @@ geometry: date: '{{ marine_window_middle_iso }}' variables: - ice: [cicen, hicen, hsnon] - ocean: [tocn, socn, uocn, vocn, ssh] + ice: + - sea_ice_area_fraction + - sea_ice_thickness + - sea_ice_snow_thickness + + ocean: + - sea_water_potential_temperature + - sea_water_salinity + - eastward_sea_water_velocity + - northward_sea_water_velocity + - sea_surface_height_above_geoid background: date: '{{ marine_window_begin_iso }}' diff --git a/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_hz.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_hz.yaml.j2 index c79d469b7..a0fc0e10b 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_hz.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_hz.yaml.j2 @@ -30,8 +30,11 @@ background error: write: filepath: ./staticb/hz_ocean - - horizontal: - as gaussian: true - fixed value: 50000.0 - write: - filepath: ./staticb/hz_ice +# TODO(G): OK to not use what's below but it will need +# to be fixed when we add ice thickness and +# snow depth +# - horizontal: +# as gaussian: true +# fixed value: 50000.0 +# write: +# filepath: ./staticb/hz_ice diff --git a/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 b/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 index 0c57790d6..4b31572f1 100644 --- a/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 +++ b/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 @@ -3,15 +3,22 @@ saber central block: saber block name: diffusion read: groups: - - variables: [sea_water_potential_temperature, sea_water_salinity, sea_surface_height_above_geoid] + - variables: + - sea_water_potential_temperature + - sea_water_salinity + - sea_surface_height_above_geoid + - sea_ice_area_fraction horizontal: filepath: ./staticb/hz_ocean vertical: levels: {{marine_vt_levels}} filepath: ./staticb/vt_ocean - - variables: [sea_ice_area_fraction] - horizontal: - filepath: ./staticb/hz_ice +# TODO(G): OK to not use what's below but it will need +# to be fixed when we add ice thickness and +# snow depth +# - variables: [sea_ice_area_fraction] +# horizontal: +# filepath: ./staticb/hz_ice saber outer blocks: - saber block name: StdDev @@ -24,7 +31,23 @@ saber outer blocks: read_from_file: 3 linear variable change: - input variables: [sea_ice_area_fraction, sea_ice_thickness, sea_ice_snow_thickness, sea_water_salinity, sea_water_potential_temperature, eastward_sea_water_velocity, northward_sea_water_velocity, sea_surface_height_above_geoid] - output variables: [sea_ice_area_fraction, sea_ice_thickness, sea_ice_snow_thickness, sea_water_salinity, sea_water_potential_temperature, eastward_sea_water_velocity, northward_sea_water_velocity, sea_surface_height_above_geoid] + input variables: + - sea_ice_area_fraction + - sea_ice_thickness + - sea_ice_snow_thickness + - sea_water_salinity + - sea_water_potential_temperature + - eastward_sea_water_velocity + - northward_sea_water_velocity + - sea_surface_height_above_geoid + output variables: + - sea_ice_area_fraction + - sea_ice_thickness + - sea_ice_snow_thickness + - sea_water_salinity + - sea_water_potential_temperature + - eastward_sea_water_velocity + - northward_sea_water_velocity + - sea_surface_height_above_geoid linear variable changes: - linear variable change name: BalanceSOCA From 8f976460756b96d5ae3e8f1d61584fd2c0e3be35 Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Thu, 12 Dec 2024 14:28:59 -0500 Subject: [PATCH 069/199] Changes to get snow 2DVar for deterministic and ensemble mean working (#52) Changes to JCB templates to allow for 2DVar snow analyses for the deterministic and the ensemble mean for GDAS. --------- Co-authored-by: Jiarui Dong Co-authored-by: yuanxue2870 <136842224+yuanxue2870@users.noreply.github.com> --- .../snow/fv3jedi_snow_ensmean.yaml.j2 | 26 ++++ .../algorithm/snow/snow_obs_staging.yaml.j2 | 16 +++ .../snow/snow_3dvar_outer_loop_1.yaml.j2 | 25 ++++ .../model/snow/snow_background.yaml.j2 | 8 ++ .../model/snow/snow_background_error.yaml.j2 | 47 +++++++ .../snow/snow_final_increment_fms.yaml.j2 | 20 +++ .../jcb-gdas/model/snow/snow_geometry.yaml.j2 | 0 .../snow/snow_geometry_background.yaml.j2 | 11 ++ .../observations/snow/adpsfc_snow.yaml.j2 | 114 +++++++++++++++++ .../observations/snow/ghcn_snow.yaml.j2 | 57 +++++++++ .../observations/snow/ims_snow.yaml.j2 | 68 ++++++++++ .../jcb-gdas/observations/snow/sfcsno.yaml.j2 | 116 ++++++++++++++++++ .../observations/snow/snocvr_snow.yaml.j2 | 109 ++++++++++++++++ 13 files changed, 617 insertions(+) create mode 100644 parm/jcb-gdas/algorithm/snow/fv3jedi_snow_ensmean.yaml.j2 create mode 100644 parm/jcb-gdas/algorithm/snow/snow_obs_staging.yaml.j2 create mode 100644 parm/jcb-gdas/model/snow/snow_3dvar_outer_loop_1.yaml.j2 create mode 100644 parm/jcb-gdas/model/snow/snow_background.yaml.j2 create mode 100644 parm/jcb-gdas/model/snow/snow_background_error.yaml.j2 create mode 100644 parm/jcb-gdas/model/snow/snow_final_increment_fms.yaml.j2 delete mode 100644 parm/jcb-gdas/model/snow/snow_geometry.yaml.j2 create mode 100644 parm/jcb-gdas/model/snow/snow_geometry_background.yaml.j2 create mode 100644 parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 create mode 100644 parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 create mode 100644 parm/jcb-gdas/observations/snow/ims_snow.yaml.j2 create mode 100644 parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 create mode 100644 parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/snow/fv3jedi_snow_ensmean.yaml.j2 b/parm/jcb-gdas/algorithm/snow/fv3jedi_snow_ensmean.yaml.j2 new file mode 100644 index 000000000..bf13c15b1 --- /dev/null +++ b/parm/jcb-gdas/algorithm/snow/fv3jedi_snow_ensmean.yaml.j2 @@ -0,0 +1,26 @@ +geometry: +{% filter indent(width=2) %} +{% set geometry_background_file = geometry_background_file|default(model_component ~ 'geometry_background', true) ~ '.yaml.j2' %} +{% include geometry_background_file %} +{% endfilter %} +mean output: + filetype: fms restart + datapath: ./bkg/ensmean + filename_core: fv_core.res.nc + filename_trcr: fv_tracer.res.nc + filename_sfcd: sfc_data.nc + filename_sfcw: fv_srf_wnd.res.nc + filename_cplr: coupler.res +ensemble: + members from template: + template: + datetime: '{{ snow_background_time_iso }}' + filetype: fms restart + state variables: [snodl,vtype,slmsk,orog_filt,stc,sheleg] + datapath: ./bkg/mem%mem%/ + filename_sfcd: '{{ snow_background_time_fv3 }}.sfc_data.nc' + filename_cplr: '{{ snow_background_time_fv3 }}.coupler.res' + filename_orog: '{{ snow_orog_prefix }}_oro_data.nc' + pattern: '%mem%' + nmembers: {{ snow_number_ensemble_members }} + zero padding: 3 diff --git a/parm/jcb-gdas/algorithm/snow/snow_obs_staging.yaml.j2 b/parm/jcb-gdas/algorithm/snow/snow_obs_staging.yaml.j2 new file mode 100644 index 000000000..a8d3dc863 --- /dev/null +++ b/parm/jcb-gdas/algorithm/snow/snow_obs_staging.yaml.j2 @@ -0,0 +1,16 @@ +mkdir: + - '{{snow_obsdatain_path}}' +copy: + {% for observation_from_jcb in observations %} + {% if use_observer(observation_from_jcb) %} + {% if not observation_from_jcb == 'ims_snow' %} + # skip if ims_snow since it is special + {% if observation_from_jcb == 'snocvr_snow' %} + # snocvr_snow is a .nc file, TODO rename this eventually? + - ['{{snow_obsdataroot_path}}/{{snow_obsdatain_prefix}}{{observation_from_jcb}}.nc4', '{{snow_obsdatain_path}}'] + {% else %} + - ['{{snow_obsdataroot_path}}/{{snow_obsdatain_prefix}}{{observation_from_jcb}}{{snow_obsdatain_suffix}}', '{{snow_obsdatain_path}}'] + {% endif %} + {% endif %} + {% endif %} + {% endfor %} diff --git a/parm/jcb-gdas/model/snow/snow_3dvar_outer_loop_1.yaml.j2 b/parm/jcb-gdas/model/snow/snow_3dvar_outer_loop_1.yaml.j2 new file mode 100644 index 000000000..48de31e89 --- /dev/null +++ b/parm/jcb-gdas/model/snow/snow_3dvar_outer_loop_1.yaml.j2 @@ -0,0 +1,25 @@ +- ninner: 50 + gradient norm reduction: 1e-10 + test: on + geometry: + fms initialization: + namelist filename: {{ snow_fv3jedi_files_path }}/fmsmpp.nml + field table filename: {{ snow_fv3jedi_files_path }}/field_table + akbk: {{ snow_fv3jedi_files_path }}/akbk.nc4 + layout: + - {{ snow_layout_x }} + - {{ snow_layout_y }} + npx: {{ snow_npx_anl }} + npy: {{ snow_npy_anl }} + npz: {{ snow_npz_anl }} + field metadata override: {{ snow_fv3jedi_files_path }}/fv3jedi_fieldmetadata_restart.yaml + time invariant fields: + state fields: + datetime: '{{ snow_background_time_iso }}' + filetype: fms restart + skip coupler file: true + state variables: [orog_filt] + datapath: {{ snow_orog_files_path }}/ + filename_orog: {{ snow_orog_prefix }}_oro_data.nc + diagnostics: + departures: bkgmob diff --git a/parm/jcb-gdas/model/snow/snow_background.yaml.j2 b/parm/jcb-gdas/model/snow/snow_background.yaml.j2 new file mode 100644 index 000000000..737d5e3f5 --- /dev/null +++ b/parm/jcb-gdas/model/snow/snow_background.yaml.j2 @@ -0,0 +1,8 @@ +datapath: {{ snow_background_path }} +filetype: fms restart +skip coupler file: true +datetime: '{{ snow_background_time_iso }}' +state variables: [snodl,vtype,slmsk,sheleg,orog_filt] +filename_sfcd: '{{ snow_background_time_fv3 }}.sfc_data.nc' +filename_cplr: '{{ snow_background_time_fv3 }}.coupler.res' +filename_orog: '{{ snow_orog_prefix }}_oro_data.nc' diff --git a/parm/jcb-gdas/model/snow/snow_background_error.yaml.j2 b/parm/jcb-gdas/model/snow/snow_background_error.yaml.j2 new file mode 100644 index 000000000..3d853038a --- /dev/null +++ b/parm/jcb-gdas/model/snow/snow_background_error.yaml.j2 @@ -0,0 +1,47 @@ +covariance model: SABER +saber central block: + saber block name: BUMP_NICAS + read: + general: + universe length-scale: 300000.0 + drivers: + multivariate strategy: univariate + read global nicas: true + nicas: + explicit length-scales: true + horizontal length-scale: + - groups: + - totalSnowDepth_shadowLevels + value: 250000.0 + vertical length-scale: + - groups: + - totalSnowDepth_shadowLevels + value: 0.0 + interpolation type: + - groups: + - totalSnowDepth_shadowLevels + type: c0 + same horizontal convolution: true + io: + data directory: {{snow_bump_data_directory}} + files prefix: snow_bump_nicas_250km_shadowlevels +saber outer blocks: +- saber block name: ShadowLevels + fields metadata: + totalSnowDepth: + vert_coord: filtered_orography + calibration: + number of shadow levels: 50 + lowest shadow level: -450.0 + highest shadow level: 8850.0 + vertical length-scale: 2000.0 +- saber block name: BUMP_StdDev + read: + drivers: + compute variance: true + variance: + explicit stddev: true + stddev: + - variables: + - totalSnowDepth + value: 30.0 diff --git a/parm/jcb-gdas/model/snow/snow_final_increment_fms.yaml.j2 b/parm/jcb-gdas/model/snow/snow_final_increment_fms.yaml.j2 new file mode 100644 index 000000000..3bd182f6b --- /dev/null +++ b/parm/jcb-gdas/model/snow/snow_final_increment_fms.yaml.j2 @@ -0,0 +1,20 @@ +output: + state component: + datapath: ./anl + prefix: snowinc + filetype: fms restart + filename_sfcd: '{{ snow_background_time_fv3 }}.sfc_data.nc' + filename_cplr: '{{ snow_background_time_fv3 }}.coupler.res' + state variables: [snodl,vtype,slmsk] +geometry: + fms initialization: + namelist filename: "{{ snow_fv3jedi_files_path }}/fmsmpp.nml" + field table filename: "{{ snow_fv3jedi_files_path }}/field_table" + akbk: "{{ snow_fv3jedi_files_path }}/akbk.nc4" + layout: + - {{ snow_layout_x }} + - {{ snow_layout_y }} + npx: {{ snow_npx_ges }} + npy: {{ snow_npy_ges }} + npz: {{ snow_npz_ges }} + field metadata override: "{{ snow_fv3jedi_files_path }}/fv3jedi_fieldmetadata_restart.yaml" diff --git a/parm/jcb-gdas/model/snow/snow_geometry.yaml.j2 b/parm/jcb-gdas/model/snow/snow_geometry.yaml.j2 deleted file mode 100644 index e69de29bb..000000000 diff --git a/parm/jcb-gdas/model/snow/snow_geometry_background.yaml.j2 b/parm/jcb-gdas/model/snow/snow_geometry_background.yaml.j2 new file mode 100644 index 000000000..e53d5ae35 --- /dev/null +++ b/parm/jcb-gdas/model/snow/snow_geometry_background.yaml.j2 @@ -0,0 +1,11 @@ +fms initialization: + namelist filename: "{{snow_fv3jedi_files_path}}/fmsmpp.nml" + field table filename: "{{snow_fv3jedi_files_path}}/field_table" +akbk: "{{snow_fv3jedi_files_path}}/akbk.nc4" +layout: +- {{snow_layout_x}} +- {{snow_layout_y}} +npx: {{snow_npx_ges}} +npy: {{snow_npy_ges}} +npz: {{snow_npz_ges}} +field metadata override: "{{snow_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" diff --git a/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 new file mode 100644 index 000000000..2f7726bc9 --- /dev/null +++ b/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 @@ -0,0 +1,114 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: adpsfc_snow + obsdatain: + engine: + type: H5File + obsfile: "{{snow_obsdatain_path}}/{{snow_obsdatain_prefix}}{{observation_from_jcb}}{{snow_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{snow_obsdataout_path}}/{{snow_obsdataout_prefix}}{{observation_from_jcb}}{{snow_obsdataout_suffix}}" + simulated variables: [totalSnowDepth] + # + + # Observation Operator + # -------------------- + obs operator: + name: Composite + components: + - name: Identity + - name: BackgroundErrorIdentity + obs error: + covariance model: diagonal + + # Observation Filters (QC) + # ------------------------ + obs pre filters: + - filter: Perform Action + filter variables: + - name: totalSnowDepth + action: + name: assign error + error parameter: 40.0 + - filter: Variable Assignment + assignments: + - name: GrossErrorProbability/totalSnowDepth + type: float + value: 0.02 + - name: BkgError/totalSnowDepth_background_error + type: float + value: 30.0 + obs prior filters: + - filter: Bounds Check + filter variables: + - name: totalSnowDepth + minvalue: 0.0 + maxvalue: 20000.0 + action: + name: reject + - filter: Domain Check + where: + - variable: + name: MetaData/stationElevation + value: is_valid + - filter: Domain Check # land only + where: + - variable: + name: GeoVaLs/slmsk + minvalue: 0.5 + maxvalue: 1.5 + - filter: RejectList # no land-ice + where: + - variable: + name: GeoVaLs/vtype + minvalue: 14.5 + maxvalue: 15.5 + - filter: Difference Check # elevation check + reference: MetaData/stationElevation + value: GeoVaLs/filtered_orography + threshold: 200. + - filter: BlackList + where: + - variable: + name: MetaData/stationIdentification + is_in: [71120,71397,71621,71727,71816] + size where true: 5 + obs post filters: + - filter: Background Check # gross error check + filter variables: + - name: totalSnowDepth + threshold: 6.25 + action: + name: reject + - filter: Temporal Thinning + min_spacing: '{{ window_length }}' + seed_time: '{{ snow_background_time_iso }}' + category_variable: + name: MetaData/stationIdentification + - filter: Met Office Buddy Check + filter variables: + - name: totalSnowDepth + rejection_threshold: 0.5 + traced_boxes: # trace all observations + min_latitude: -90 + max_latitude: 90 + min_longitude: -180 + max_longitude: 180 + search_radius: 150 # km + station_id_variable: + name: MetaData/stationIdentification + num_zonal_bands: 24 + sort_by_pressure: false + max_total_num_buddies: 15 + max_num_buddies_from_single_band: 10 + max_num_buddies_with_same_station_id: 5 + use_legacy_buddy_collector: false + horizontal_correlation_scale: { "-90": 150, "90": 150 } + temporal_correlation_scale: PT6H + damping_factor_1: 1.0 + damping_factor_2: 1.0 + background_error_group: BkgError diff --git a/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 new file mode 100644 index 000000000..37022cb55 --- /dev/null +++ b/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 @@ -0,0 +1,57 @@ + - + + # Observation Space (I/O) + # ----------------------- + obs space: + name: ghcn_snow + obsdatain: + engine: + type: H5File + obsfile: "{{snow_obsdatain_path}}/{{snow_obsdatain_prefix}}{{observation_from_jcb}}.nc" + obsdataout: + engine: + type: H5File + obsfile: "{{snow_obsdataout_path}}/{{snow_obsdataout_prefix}}{{observation_from_jcb}}{{snow_obsdataout_suffix}}" + simulated variables: [totalSnowDepth] + # + + # Observation Operator + # -------------------- + obs operator: + name: Identity + # + + # Observation Filters (QC) + # ------------------------ + obs filters: + - filter: Bounds Check + filter variables: + - name: totalSnowDepth + minvalue: 0.0 + - filter: Domain Check + where: + - variable: + name: MetaData/stationElevation + value: is_valid + - filter: Domain Check # land only + where: + - variable: + name: GeoVaLs/slmsk + minvalue: 0.5 + maxvalue: 1.5 + - filter: RejectList # no land-ice + where: + - variable: + name: GeoVaLs/vtype + minvalue: 14.5 + maxvalue: 15.5 + - filter: Difference Check # elevation check + reference: MetaData/stationElevation + value: GeoVaLs/filtered_orography + threshold: 200. + - filter: Background Check + filter variables: + - name: totalSnowDepth + threshold: 6.25 + action: + name: reject diff --git a/parm/jcb-gdas/observations/snow/ims_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/ims_snow.yaml.j2 new file mode 100644 index 000000000..612f2d73a --- /dev/null +++ b/parm/jcb-gdas/observations/snow/ims_snow.yaml.j2 @@ -0,0 +1,68 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: ims_snow + obsdatain: + engine: + type: H5File + obsfile: "{{snow_obsdatain_path}}/{{snow_obsdatain_prefix}}{{observation_from_jcb}}.tm00.nc" + missing file action: warn + obsdataout: + engine: + type: H5File + obsfile: "{{snow_obsdataout_path}}/{{snow_obsdataout_prefix}}{{observation_from_jcb}}{{snow_obsdataout_suffix}}" + simulated variables: [totalSnowDepth] + # + + # Observation Operator + # -------------------- + obs operator: + name: Identity + # + + # Observation Filters (QC) + # ------------------------ + obs pre filters: + # assign observation error + - filter: Perform Action + filter variables: + - name: totalSnowDepth + action: + name: assign error + error parameter: 80.0 + + obs prior filters: + # remove land-ice and sea points + - filter: Domain Check # land only + where: + - variable: + name: GeoVaLs/slmsk + minvalue: 0.5 + maxvalue: 1.5 + - filter: RejectList # no land-ice + where: + - variable: + name: GeoVaLs/vtype + minvalue: 14.5 + maxvalue: 15.5 + + obs post filters: + # gross error check + - filter: Background Check + filter variables: + - name: totalSnowDepth + threshold: 3. + action: + name: reject + + # thinning of remaining obs + - filter: Gaussian Thinning + horizontal_mesh: 40.0 # km + + # excludes where both obs & mod = 100% (set in IMS_proc) + - filter: Bounds Check + filter variables: + - name: totalSnowDepth + minvalue: 0.0 diff --git a/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 b/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 new file mode 100644 index 000000000..a8f340ae8 --- /dev/null +++ b/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 @@ -0,0 +1,116 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: sfcsno + obsdatain: + engine: + type: bufr + obsfile: "{{snow_obsdatain_path}}/{{snow_obsdatain_prefix}}sfcsno.tm00.bufr_d" + mapping file: "{{snow_obsdatain_path}}/bufr_sfcsno_mapping.yaml" + obsdataout: + engine: + type: H5File + obsfile: "{{snow_obsdataout_path}}/{{snow_obsdataout_prefix}}sfcsno{{snow_obsdataout_suffix}}" + simulated variables: [totalSnowDepth] + # + + # Observation Operator + # -------------------- + obs operator: + name: Composite + components: + - name: Identity + - name: BackgroundErrorIdentity + linear obs operator: + name: Identity + # + + # Observation Filters (QC) + # ------------------------ + obs pre filters: + - filter: Perform Action + filter variables: + - name: totalSnowDepth + action: + name: assign error + error parameter: 40.0 + - filter: Variable Assignment + assignments: + - name: GrossErrorProbability/totalSnowDepth + type: float + value: 0.02 + - name: BkgError/totalSnowDepth_background_error + type: float + value: 30.0 + obs prior filters: + - filter: Bounds Check + filter variables: + - name: totalSnowDepth + minvalue: 0.0 + maxvalue: 20000.0 + action: + name: reject + - filter: Domain Check + where: + - variable: + name: MetaData/stationElevation + value: is_valid + - filter: Domain Check # land only + where: + - variable: + name: GeoVaLs/slmsk + minvalue: 0.5 + maxvalue: 1.5 + - filter: RejectList # no land-ice + where: + - variable: + name: GeoVaLs/vtype + minvalue: 14.5 + maxvalue: 15.5 + - filter: Difference Check # elevation check + reference: MetaData/stationElevation + value: GeoVaLs/filtered_orography + threshold: 200. + - filter: BlackList + where: + - variable: + name: MetaData/stationIdentification + is_in: [71120,71397,71621,71727,71816] + size where true: 5 + obs post filters: + - filter: Background Check # gross error check + filter variables: + - name: totalSnowDepth + threshold: 6.25 + action: + name: reject + - filter: Temporal Thinning + min_spacing: '{{ window_length }}' + seed_time: '{{ snow_background_time_iso }}' + category_variable: + name: MetaData/stationIdentification + - filter: Met Office Buddy Check + filter variables: + - name: totalSnowDepth + rejection_threshold: 0.5 + traced_boxes: # trace all observations + min_latitude: -90 + max_latitude: 90 + min_longitude: -180 + max_longitude: 180 + search_radius: 150 # km + station_id_variable: + name: MetaData/stationIdentification + num_zonal_bands: 24 + sort_by_pressure: false + max_total_num_buddies: 15 + max_num_buddies_from_single_band: 10 + max_num_buddies_with_same_station_id: 5 + use_legacy_buddy_collector: false + horizontal_correlation_scale: { "-90": 150, "90": 150 } + temporal_correlation_scale: PT6H + damping_factor_1: 1.0 + damping_factor_2: 1.0 + background_error_group: BkgError diff --git a/parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 new file mode 100644 index 000000000..b072f7f6b --- /dev/null +++ b/parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 @@ -0,0 +1,109 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: snocvr_snow + obsdatain: + engine: + type: H5File + obsfile: "{{snow_obsdatain_path}}/{{snow_obsdatain_prefix}}{{observation_from_jcb}}.nc4" + obsdataout: + engine: + type: H5File + obsfile: "{{snow_obsdataout_path}}/{{snow_obsdataout_prefix}}{{observation_from_jcb}}{{snow_obsdataout_suffix}}" + simulated variables: [totalSnowDepth] + # + + # Observation Operator + # -------------------- + obs operator: + name: Composite + components: + - name: Identity + - name: BackgroundErrorIdentity + linear obs operator: + name: Identity + # + + # Observation Filters (QC) + # ------------------------ + obs pre filters: + - filter: Perform Action + filter variables: + - name: totalSnowDepth + action: + name: assign error + error parameter: 40.0 + - filter: Variable Assignment + assignments: + - name: GrossErrorProbability/totalSnowDepth + type: float + value: 0.02 + - name: BkgError/totalSnowDepth_background_error + type: float + value: 30.0 + obs prior filters: + - filter: Bounds Check + filter variables: + - name: totalSnowDepth + minvalue: 0.0 + maxvalue: 20000.0 + action: + name: reject + - filter: Domain Check + where: + - variable: + name: MetaData/stationElevation + value: is_valid + - filter: Domain Check # land only + where: + - variable: + name: GeoVaLs/slmsk + minvalue: 0.5 + maxvalue: 1.5 + - filter: RejectList # no land-ice + where: + - variable: + name: GeoVaLs/vtype + minvalue: 14.5 + maxvalue: 15.5 + - filter: Difference Check # elevation check + reference: MetaData/stationElevation + value: GeoVaLs/filtered_orography + threshold: 200. + obs post filters: + - filter: Background Check # gross error check + filter variables: + - name: totalSnowDepth + threshold: 6.25 + action: + name: reject + - filter: Temporal Thinning + min_spacing: '{{ window_length }}' + seed_time: '{{ snow_background_time_iso }}' + category_variable: + name: MetaData/stationIdentification + - filter: Met Office Buddy Check + filter variables: + - name: totalSnowDepth + rejection_threshold: 0.5 + traced_boxes: # trace all observations + min_latitude: -90 + max_latitude: 90 + min_longitude: -180 + max_longitude: 180 + search_radius: 150 # km + station_id_variable: + name: MetaData/stationIdentification + num_zonal_bands: 24 + sort_by_pressure: false + max_total_num_buddies: 15 + max_num_buddies_from_single_band: 10 + max_num_buddies_with_same_station_id: 5 + use_legacy_buddy_collector: false + horizontal_correlation_scale: { "-90": 150, "90": 150 } + temporal_correlation_scale: PT6H + damping_factor_1: 1.0 + damping_factor_2: 1.0 + background_error_group: BkgError From ce50d9ed73d834e792b98a88267c379c050a014e Mon Sep 17 00:00:00 2001 From: Jiarui Dong Date: Fri, 13 Dec 2024 09:29:40 -0500 Subject: [PATCH 070/199] Fix the ParserError due to the indent spaces. (#54) This PR fixes the issues with yaml format for the `Difference Check` filter. --- parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 | 6 +++--- parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 | 6 +++--- parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 | 6 +++--- parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 | 6 +++--- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 index 2f7726bc9..f0fe2fd42 100644 --- a/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 @@ -68,9 +68,9 @@ minvalue: 14.5 maxvalue: 15.5 - filter: Difference Check # elevation check - reference: MetaData/stationElevation - value: GeoVaLs/filtered_orography - threshold: 200. + reference: MetaData/stationElevation + value: GeoVaLs/filtered_orography + threshold: 200. - filter: BlackList where: - variable: diff --git a/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 index 37022cb55..ea7a95655 100644 --- a/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 @@ -46,9 +46,9 @@ minvalue: 14.5 maxvalue: 15.5 - filter: Difference Check # elevation check - reference: MetaData/stationElevation - value: GeoVaLs/filtered_orography - threshold: 200. + reference: MetaData/stationElevation + value: GeoVaLs/filtered_orography + threshold: 200. - filter: Background Check filter variables: - name: totalSnowDepth diff --git a/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 b/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 index a8f340ae8..f1ef5d4e8 100644 --- a/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 @@ -70,9 +70,9 @@ minvalue: 14.5 maxvalue: 15.5 - filter: Difference Check # elevation check - reference: MetaData/stationElevation - value: GeoVaLs/filtered_orography - threshold: 200. + reference: MetaData/stationElevation + value: GeoVaLs/filtered_orography + threshold: 200. - filter: BlackList where: - variable: diff --git a/parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 index b072f7f6b..29ba1595e 100644 --- a/parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 @@ -69,9 +69,9 @@ minvalue: 14.5 maxvalue: 15.5 - filter: Difference Check # elevation check - reference: MetaData/stationElevation - value: GeoVaLs/filtered_orography - threshold: 200. + reference: MetaData/stationElevation + value: GeoVaLs/filtered_orography + threshold: 200. obs post filters: - filter: Background Check # gross error check filter variables: From 134b67834c785e3a0393b3ed97d23cc982d077e8 Mon Sep 17 00:00:00 2001 From: Kevin Dougherty <69815622+kevindougherty-noaa@users.noreply.github.com> Date: Fri, 13 Dec 2024 10:16:59 -0500 Subject: [PATCH 071/199] Add obspace templates for stat analysis job (#53) Add obspace templates for stat analysis jobs run in global-workflow. --------- Co-authored-by: Dan Holdaway <27729500+danholdaway@users.noreply.github.com> --- .../obstats/aero/viirs_n20_template.yaml.j2 | 50 +++++++++++++++++++ .../obstats/aero/viirs_npp_template.yaml.j2 | 50 +++++++++++++++++++ 2 files changed, 100 insertions(+) create mode 100644 parm/jcb-gdas/algorithm/obstats/aero/viirs_n20_template.yaml.j2 create mode 100644 parm/jcb-gdas/algorithm/obstats/aero/viirs_npp_template.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/obstats/aero/viirs_n20_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/aero/viirs_n20_template.yaml.j2 new file mode 100644 index 000000000..9606dd922 --- /dev/null +++ b/parm/jcb-gdas/algorithm/obstats/aero/viirs_n20_template.yaml.j2 @@ -0,0 +1,50 @@ +- obs space: + name: aod + obsdatain: + engine: + type: H5File + obsfile: {{ aero_obsdatain_path }}/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc4 + simulated variables: {{ aero_obsdatatin_simulated_variables }} + observed variables: {{ aero_obsdatain_observed_variables }} + variables: {{ aero_variables }} + groups to process: {{ aero_file_groups }} + qc groups: {{ aero_file_qc_groups }} + statistics to compute: ['mean', 'count', 'RMS'] + output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_aod.nc" + domains to process: + - domain: + name: "nowhere" + first mask variable: latitude + first mask range: [-180,-100] + - domain: + name: "SH" + first mask variable: latitude + first mask range: [-90,0] + - domain: + name: "NH" + first mask variable: latitude + first mask range: [0,90] + - domain: + name: "CONUS" + first mask variable: latitude + first mask range: [25,49] + second mask variable: longitude + second mask range: [-125,-66] + - domain: + name: "Europe" + first mask variable: latitude + first mask range: [35,70] + second mask variable: longitude + second mask range: [-11,38] + - domain: + name: "Africa" + first mask variable: latitude + first mask range: [-35,37] + second mask variable: longitude + second mask range: [-17,52] + - domain: + name: "Asia" + first mask variable: latitude + first mask range: [0,70] + second mask variable: longitude + second mask range: [38, 180] diff --git a/parm/jcb-gdas/algorithm/obstats/aero/viirs_npp_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/aero/viirs_npp_template.yaml.j2 new file mode 100644 index 000000000..e0909bbbf --- /dev/null +++ b/parm/jcb-gdas/algorithm/obstats/aero/viirs_npp_template.yaml.j2 @@ -0,0 +1,50 @@ +- obs space: + name: aod + obsdatain: + engine: + type: H5File + obsfile: {{ aero_obsdatain_path }}/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc4 + simulated variables: {{ aero_obsdatatin_simulated_variables }} + observed variables: {{ aero_obsdatain_observed_variables }} + variables: {{ aero_variables }} + groups to process: {{ aero_file_groups }} + qc groups: {{ aero_file_qc_groups }} + statistics to compute: ['mean', 'count', 'RMS'] + output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_aod.nc" + domains to process: + - domain: + name: "nowhere" + first mask variable: latitude + first mask range: [-180,-100] + - domain: + name: "SH" + first mask variable: latitude + first mask range: [-90,0] + - domain: + name: "NH" + first mask variable: latitude + first mask range: [0,90] + - domain: + name: "CONUS" + first mask variable: latitude + first mask range: [25,49] + second mask variable: longitude + second mask range: [-125,-66] + - domain: + name: "Europe" + first mask variable: latitude + first mask range: [35,70] + second mask variable: longitude + second mask range: [-11,38] + - domain: + name: "Africa" + first mask variable: latitude + first mask range: [-35,37] + second mask variable: longitude + second mask range: [-17,52] + - domain: + name: "Asia" + first mask variable: latitude + first mask range: [0,70] + second mask variable: longitude + second mask range: [38, 180] \ No newline at end of file From 0c238b1c8c3387470d9d08bba927f8811936cc39 Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Fri, 13 Dec 2024 13:56:41 -0500 Subject: [PATCH 072/199] write out aerosol analysis for GEFS + ensure proper increment resolution for cycling (#55) This PR will write out a separate aerosol analysis on the analysis resolution to be picked up by the GEFS aerosol member, it also corrects the resolution of the increment write for cycling which should be the background resolution, not the analysis resolution. --------- Co-authored-by: Cory Martin --- .../model/aero/aero_final_increment_cubed_sphere.yaml.j2 | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/parm/jcb-gdas/model/aero/aero_final_increment_cubed_sphere.yaml.j2 b/parm/jcb-gdas/model/aero/aero_final_increment_cubed_sphere.yaml.j2 index e34a7fcfa..d489b2b70 100644 --- a/parm/jcb-gdas/model/aero/aero_final_increment_cubed_sphere.yaml.j2 +++ b/parm/jcb-gdas/model/aero/aero_final_increment_cubed_sphere.yaml.j2 @@ -6,9 +6,9 @@ geometry: layout: - {{aero_layout_x}} - {{aero_layout_y}} - npx: {{aero_npx_anl}} - npy: {{aero_npy_anl}} - npz: {{aero_npz_anl}} + npx: {{aero_npx_ges}} + npy: {{aero_npy_ges}} + npz: {{aero_npz_ges}} field metadata override: "{{aero_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" output: state component: From 55d79e7d59b314f18e9a9e344eebe6bd5e086ef9 Mon Sep 17 00:00:00 2001 From: Guillaume Vernieres Date: Mon, 16 Dec 2024 13:32:40 -0500 Subject: [PATCH 073/199] Updated configuration for "cp4" (#58) - Added filtering to the variance partitioning - Increased decorrelation length scales - Removed u,v from state/incr - Fixed outdated hybrid B yaml - Updated obs configuration from @JohnSteffen-NOAA --------- Co-authored-by: JohnSteffen-NOAA <77806417+JohnSteffen-NOAA@users.noreply.github.com> --- .../algorithm/marine/soca_diagb.yaml.j2 | 2 +- .../soca_parameters_diffusion_hz.yaml.j2 | 2 +- .../marine/soca_setcorscales.yaml.j2 | 4 +- .../algorithm/marine/socaincr2mom6.yaml.j2 | 11 +++- .../model/marine/marine_background.yaml.j2 | 16 ++++- ...d_error_hybrid_diffusion_diffusion.yaml.j2 | 59 ++++++++++++++----- ..._background_error_static_diffusion.yaml.j2 | 26 ++++++-- .../observations/marine/adt_rads_all.yaml.j2 | 2 +- .../marine/icec_abi_g16_l2.yaml.j2 | 12 ++++ .../marine/icec_amsr2_north.yaml.j2 | 6 ++ .../marine/icec_amsr2_south.yaml.j2 | 6 ++ .../marine/icec_amsu_mb_l2.yaml.j2 | 12 ++++ .../marine/icec_atms_n20_l2.yaml.j2 | 12 ++++ .../marine/icec_atms_n21_l2.yaml.j2 | 12 ++++ .../marine/icec_atms_npp_l2.yaml.j2 | 12 ++++ .../marine/icec_gmi_gpm_l2.yaml.j2 | 6 ++ .../marine/icec_ssmis_f17_l2.yaml.j2 | 12 ++++ .../marine/icec_viirs_n20_l2_north.yaml.j2 | 12 ++++ .../marine/icec_viirs_n20_l2_south.yaml.j2 | 12 ++++ .../observations/marine/sss_smap_l2.yaml.j2 | 3 +- .../observations/marine/sss_smos_l2.yaml.j2 | 3 +- 21 files changed, 215 insertions(+), 27 deletions(-) diff --git a/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 index 30e9c6864..788f674de 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 @@ -39,7 +39,7 @@ number of halo points: 4 number of neighbors: 16 simple smoothing: - horizontal iterations: 10 + horizontal iterations: 100 vertical iterations: 1 # TODO(G): Too slow for the below scale diff --git a/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_hz.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_hz.yaml.j2 index a0fc0e10b..8f3cdf798 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_hz.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_hz.yaml.j2 @@ -18,7 +18,7 @@ background error: calibration: normalization: method: randomization - iterations: 10000 + iterations: 1000 groups: - horizontal: diff --git a/parm/jcb-gdas/algorithm/marine/soca_setcorscales.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_setcorscales.yaml.j2 index c1a1ec86c..0c2b9c06b 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_setcorscales.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_setcorscales.yaml.j2 @@ -9,8 +9,8 @@ corr variables: [sea_surface_height_above_geoid] scales: vert layers: 5 # in units of layer sea_surface_height_above_geoid: - rossby mult: 1.00 - min grid mult: 2.0 + rossby mult: 2.00 + min grid mult: 6.0 rh output: datadir: ./ diff --git a/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 b/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 index deaec761a..0790fc9c5 100644 --- a/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 @@ -6,7 +6,16 @@ date: '{{ marine_window_begin }}' layers variable: [sea_water_cell_thickness] -increment variables: [sea_water_potential_temperature, sea_water_salinity, eastward_sea_water_velocity, northward_sea_water_velocity, sea_surface_height_above_geoid] +increment variables: +- sea_water_potential_temperature +- sea_water_salinity +#- eastward_sea_water_velocity +#- northward_sea_water_velocity +- sea_surface_height_above_geoid # Not used in MOM6 iau + +set increment variables to zero: +- eastward_sea_water_velocity +- northward_sea_water_velocity vertical geometry: date: '{{ marine_window_begin }}' diff --git a/parm/jcb-gdas/model/marine/marine_background.yaml.j2 b/parm/jcb-gdas/model/marine/marine_background.yaml.j2 index 205f58337..4be1f74fc 100644 --- a/parm/jcb-gdas/model/marine/marine_background.yaml.j2 +++ b/parm/jcb-gdas/model/marine/marine_background.yaml.j2 @@ -3,4 +3,18 @@ basename: {{ marine_background_path}} ocn_filename: MOM.res.nc ice_filename: cice.res.nc date: "{{ marine_background_time }}" -state variables: [sea_ice_area_fraction, sea_ice_thickness, sea_ice_snow_thickness, sea_water_salinity, sea_water_potential_temperature, eastward_sea_water_velocity, northward_sea_water_velocity, sea_surface_height_above_geoid, sea_water_cell_thickness, ocean_mixed_layer_thickness, sea_water_depth] +state variables: +- sea_ice_area_fraction +- sea_ice_thickness +- sea_ice_snow_thickness +- snow_ice_surface_temperature +- air_temperature +- bulk_ice_salinity +- sea_water_salinity +- sea_water_potential_temperature +#- eastward_sea_water_velocity +#- northward_sea_water_velocity +- sea_surface_height_above_geoid +- sea_water_cell_thickness +- ocean_mixed_layer_thickness +- sea_water_depth diff --git a/parm/jcb-gdas/model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 b/parm/jcb-gdas/model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 index b2709aafb..5a5a1ac11 100644 --- a/parm/jcb-gdas/model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 +++ b/parm/jcb-gdas/model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 @@ -6,15 +6,22 @@ components: saber block name: diffusion read: groups: - - variables: [tocn, socn, ssh] + - variables: + - sea_water_potential_temperature + - sea_water_salinity + - sea_surface_height_above_geoid + - sea_ice_area_fraction horizontal: filepath: ./staticb/hz_ocean vertical: levels: {{marine_vt_levels}} filepath: ./staticb/vt_ocean - - variables: [cicen] - horizontal: - filepath: ./staticb/hz_ice + # TODO(G): OK to not use what's below but it will need + # to be fixed when we add ice thickness and + # snow depth + # - variables: [sea_ice_area_fraction] + # horizontal: + # filepath: ./staticb/hz_ice saber outer blocks: - saber block name: StdDev @@ -25,10 +32,25 @@ components: ocn_filename: 'ocn.bkgerr_stddev.nc' ice_filename: 'ice.bkgerr_stddev.nc' read_from_file: 3 - linear variable change: - input variables: [cicen, hicen, hsnon, socn, tocn, uocn, vocn, ssh] - output variables: [cicen, hicen, hsnon, socn, tocn, uocn, vocn, ssh] + input variables: + - sea_ice_area_fraction + - sea_ice_thickness + - sea_ice_snow_thickness + - sea_water_salinity + - sea_water_potential_temperature + #- eastward_sea_water_velocity + #- northward_sea_water_velocity + - sea_surface_height_above_geoid + output variables: + - sea_ice_area_fraction + - sea_ice_thickness + - sea_ice_snow_thickness + - sea_water_salinity + - sea_water_potential_temperature + #- eastward_sea_water_velocity + #- northward_sea_water_velocity + - sea_surface_height_above_geoid linear variable changes: - linear variable change name: BalanceSOCA weight: @@ -43,7 +65,15 @@ components: basename: ../ensdata/ ocn_filename: 'ocn.pert.steric.%mem%.nc' ice_filename: 'ens/ice.%mem%.nc' - state variables: [tocn, socn, ssh, uocn, vocn, cicen, hicen, hsnon] + state variables: + - sea_ice_area_fraction + - sea_ice_thickness + - sea_ice_snow_thickness + - sea_water_salinity + - sea_water_potential_temperature + #- eastward_sea_water_velocity + #- northward_sea_water_velocity + - sea_surface_height_above_geoid pattern: '%mem%' nmembers: '{{marine_number_ensemble_members}}' localization: @@ -52,15 +82,16 @@ components: saber block name: diffusion read: groups: - - variables: [tocn, socn, ssh] - multivariate strategy: duplicated + - variables: + - sea_water_potential_temperature + - sea_water_salinity + - sea_surface_height_above_geoid + - sea_ice_area_fraction horizontal: filepath: ./staticb/hz_ocean vertical: - strategy: duplicated - - variables: [cicen] - horizontal: - filepath: ./staticb/hz_ice + levels: {{marine_vt_levels}} + filepath: ./staticb/vt_ocean weight: read_from_file: 3 basename: ./ diff --git a/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 b/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 index 4b31572f1..2a41011d6 100644 --- a/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 +++ b/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 @@ -30,6 +30,24 @@ saber outer blocks: ice_filename: 'ice.bkgerr_stddev.nc' read_from_file: 3 +#- saber block name: MLBalance +# geometry: +# mom6_input_nml: mom_input.nml +# fields metadata: ./fields_metadata.yaml +# ML Balances: +# arctic: +# ffnn: +# inputSize: 7 +# outputSize: 1 +# hiddenSize: 2 +# load model: /work/noaa/da/gvernier/ai/runs/arctic.pt +# antarctic: +# ffnn: +# inputSize: 7 +# outputSize: 1 +# hiddenSize: 2 +# load model: /work/noaa/da/gvernier/ai/runs/antarctic.pt + linear variable change: input variables: - sea_ice_area_fraction @@ -37,8 +55,8 @@ linear variable change: - sea_ice_snow_thickness - sea_water_salinity - sea_water_potential_temperature - - eastward_sea_water_velocity - - northward_sea_water_velocity + #- eastward_sea_water_velocity + #- northward_sea_water_velocity - sea_surface_height_above_geoid output variables: - sea_ice_area_fraction @@ -46,8 +64,8 @@ linear variable change: - sea_ice_snow_thickness - sea_water_salinity - sea_water_potential_temperature - - eastward_sea_water_velocity - - northward_sea_water_velocity + #- eastward_sea_water_velocity + #- northward_sea_water_velocity - sea_surface_height_above_geoid linear variable changes: - linear variable change name: BalanceSOCA diff --git a/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 b/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 index e3be83cfb..1fc2f054e 100644 --- a/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 @@ -41,7 +41,7 @@ variables: [GeoVaLs/mesoscale_representation_error, ObsError/absoluteDynamicTopography] coefs: [0.2, - 2.0] + 10.0] - filter: Domain Check where: - variable: { name: GeoVaLs/sea_ice_area_fraction} diff --git a/parm/jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 index c63389837..33eeb2382 100644 --- a/parm/jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 @@ -43,3 +43,15 @@ where: - variable: {name: GeoVaLs/distance_from_coast} minvalue: 100e3 + - filter: Gaussian Thinning + horizontal_mesh: 25 #km + use_reduced_horizontal_grid: true + select_median: true + action: + name: reduce obs space + - filter: Domain Check + action: + name: passivate + where: + - variable: {name: GeoVaLs/sea_surface_temperature} + maxvalue: -4.0 diff --git a/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 index c63389837..6d88cc36f 100644 --- a/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 @@ -43,3 +43,9 @@ where: - variable: {name: GeoVaLs/distance_from_coast} minvalue: 100e3 +# - filter: Gaussian Thinning +# horizontal_mesh: 25 #km +# use_reduced_horizontal_grid: true +# select_median: true +# action: +# name: reduce obs space diff --git a/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 index c63389837..6d88cc36f 100644 --- a/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 @@ -43,3 +43,9 @@ where: - variable: {name: GeoVaLs/distance_from_coast} minvalue: 100e3 +# - filter: Gaussian Thinning +# horizontal_mesh: 25 #km +# use_reduced_horizontal_grid: true +# select_median: true +# action: +# name: reduce obs space diff --git a/parm/jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 index c63389837..33eeb2382 100644 --- a/parm/jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 @@ -43,3 +43,15 @@ where: - variable: {name: GeoVaLs/distance_from_coast} minvalue: 100e3 + - filter: Gaussian Thinning + horizontal_mesh: 25 #km + use_reduced_horizontal_grid: true + select_median: true + action: + name: reduce obs space + - filter: Domain Check + action: + name: passivate + where: + - variable: {name: GeoVaLs/sea_surface_temperature} + maxvalue: -4.0 diff --git a/parm/jcb-gdas/observations/marine/icec_atms_n20_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_atms_n20_l2.yaml.j2 index c63389837..33eeb2382 100644 --- a/parm/jcb-gdas/observations/marine/icec_atms_n20_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_atms_n20_l2.yaml.j2 @@ -43,3 +43,15 @@ where: - variable: {name: GeoVaLs/distance_from_coast} minvalue: 100e3 + - filter: Gaussian Thinning + horizontal_mesh: 25 #km + use_reduced_horizontal_grid: true + select_median: true + action: + name: reduce obs space + - filter: Domain Check + action: + name: passivate + where: + - variable: {name: GeoVaLs/sea_surface_temperature} + maxvalue: -4.0 diff --git a/parm/jcb-gdas/observations/marine/icec_atms_n21_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_atms_n21_l2.yaml.j2 index c63389837..33eeb2382 100644 --- a/parm/jcb-gdas/observations/marine/icec_atms_n21_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_atms_n21_l2.yaml.j2 @@ -43,3 +43,15 @@ where: - variable: {name: GeoVaLs/distance_from_coast} minvalue: 100e3 + - filter: Gaussian Thinning + horizontal_mesh: 25 #km + use_reduced_horizontal_grid: true + select_median: true + action: + name: reduce obs space + - filter: Domain Check + action: + name: passivate + where: + - variable: {name: GeoVaLs/sea_surface_temperature} + maxvalue: -4.0 diff --git a/parm/jcb-gdas/observations/marine/icec_atms_npp_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_atms_npp_l2.yaml.j2 index c63389837..33eeb2382 100644 --- a/parm/jcb-gdas/observations/marine/icec_atms_npp_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_atms_npp_l2.yaml.j2 @@ -43,3 +43,15 @@ where: - variable: {name: GeoVaLs/distance_from_coast} minvalue: 100e3 + - filter: Gaussian Thinning + horizontal_mesh: 25 #km + use_reduced_horizontal_grid: true + select_median: true + action: + name: reduce obs space + - filter: Domain Check + action: + name: passivate + where: + - variable: {name: GeoVaLs/sea_surface_temperature} + maxvalue: -4.0 diff --git a/parm/jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 index c63389837..4018e17f2 100644 --- a/parm/jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 @@ -43,3 +43,9 @@ where: - variable: {name: GeoVaLs/distance_from_coast} minvalue: 100e3 + - filter: Domain Check + action: + name: passivate + where: + - variable: {name: GeoVaLs/sea_surface_temperature} + maxvalue: -4.0 diff --git a/parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 index c63389837..33eeb2382 100644 --- a/parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 @@ -43,3 +43,15 @@ where: - variable: {name: GeoVaLs/distance_from_coast} minvalue: 100e3 + - filter: Gaussian Thinning + horizontal_mesh: 25 #km + use_reduced_horizontal_grid: true + select_median: true + action: + name: reduce obs space + - filter: Domain Check + action: + name: passivate + where: + - variable: {name: GeoVaLs/sea_surface_temperature} + maxvalue: -4.0 diff --git a/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_north.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_north.yaml.j2 index c63389837..33eeb2382 100644 --- a/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_north.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_north.yaml.j2 @@ -43,3 +43,15 @@ where: - variable: {name: GeoVaLs/distance_from_coast} minvalue: 100e3 + - filter: Gaussian Thinning + horizontal_mesh: 25 #km + use_reduced_horizontal_grid: true + select_median: true + action: + name: reduce obs space + - filter: Domain Check + action: + name: passivate + where: + - variable: {name: GeoVaLs/sea_surface_temperature} + maxvalue: -4.0 diff --git a/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_south.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_south.yaml.j2 index c63389837..33eeb2382 100644 --- a/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_south.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_south.yaml.j2 @@ -43,3 +43,15 @@ where: - variable: {name: GeoVaLs/distance_from_coast} minvalue: 100e3 + - filter: Gaussian Thinning + horizontal_mesh: 25 #km + use_reduced_horizontal_grid: true + select_median: true + action: + name: reduce obs space + - filter: Domain Check + action: + name: passivate + where: + - variable: {name: GeoVaLs/sea_surface_temperature} + maxvalue: -4.0 diff --git a/parm/jcb-gdas/observations/marine/sss_smap_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/sss_smap_l2.yaml.j2 index c8b50b2d2..20b645b2f 100644 --- a/parm/jcb-gdas/observations/marine/sss_smap_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sss_smap_l2.yaml.j2 @@ -33,7 +33,8 @@ name: passivate where: - variable: {name: GeoVaLs/sea_surface_temperature} - minvalue: 15.0 +# minvalue: 15.0 + maxvalue: -4.0 ## Gaussian_Thinning is having problems with LETKF, try again later # - filter: Gaussian_Thinning # horizontal_mesh: 25.0 #km diff --git a/parm/jcb-gdas/observations/marine/sss_smos_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/sss_smos_l2.yaml.j2 index c8b50b2d2..20b645b2f 100644 --- a/parm/jcb-gdas/observations/marine/sss_smos_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sss_smos_l2.yaml.j2 @@ -33,7 +33,8 @@ name: passivate where: - variable: {name: GeoVaLs/sea_surface_temperature} - minvalue: 15.0 +# minvalue: 15.0 + maxvalue: -4.0 ## Gaussian_Thinning is having problems with LETKF, try again later # - filter: Gaussian_Thinning # horizontal_mesh: 25.0 #km From 8984082abf28c40aebb5368d17f81028c9aec893 Mon Sep 17 00:00:00 2001 From: Anna Shlyaeva Date: Wed, 18 Dec 2024 11:55:41 -0700 Subject: [PATCH 074/199] Turn shuffle option on in soca -> cice. (#60) This turns on shuffle option in soca -> cice and changes `seaice edge` to 40%. For all background ice concentrations < `seaice edge` when shuffle is on, the code finds a nearest background point whose total ice concentration is closest to the soca analysis total ice concentration, and copies that background ice profile as the new analysis at the current point. For all background ice concentrations > `seaice edge` ice concentrations are rescaled, and some ice fields recomputed. Some results from a first cycle of high res experiment: Background: ![ice_bg](https://github.com/user-attachments/assets/645f8826-9cad-4987-8fc4-7cf240ee5b15) soca analysis (no postprocessing; I am cutting off values outside of [0, 1] when plotting): ![ice_an_soca](https://github.com/user-attachments/assets/e52651ea-e30f-4091-bf88-2becb8ee6c4c) analysis after Soca2Cice on develop (where `shuffle` is off, and `seaice edge` for the arctic is 0.8) As expected, the analysis looks like background for ice concentrations < 0.8 ![ice_an_soca2cice_noshuffle](https://github.com/user-attachments/assets/2ba397d5-14a9-451f-bbeb-b5bae4e3c20d) analysis after Soca2Cice in this branch (`shuffle` is on, `seaice edge` is set to 0.4). Note that these results are with https://github.com/JCSDA-internal/soca/pull/1108 ![ice_an_soca2cice_shuffle_bugfix_edge40](https://github.com/user-attachments/assets/4b45072c-1ea3-47f6-a488-62f7720bd1fa) This branch can be merged in develop regardless of https://github.com/JCSDA-internal/soca/pull/1108, since gdasapp soca hash already includes the bugfix that allows `shuffle` to be turned on. However, I would recommend running experiments that include https://github.com/JCSDA-internal/soca/pull/1108 which further improves on shuffle. I ran C384mx025_3DVarAOWCDA for 2 cycles with this setup, and forecasts succeeded (cice was OK with these analyses). We'll do better evaluation when we run the next marine candidate experiment. Co-authored-by: Anna Shlyaeva --- parm/jcb-gdas/algorithm/marine/soca_2cice_global.yaml.j2 | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/parm/jcb-gdas/algorithm/marine/soca_2cice_global.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_2cice_global.yaml.j2 index 8d13d46b0..01bff96dd 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_2cice_global.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_2cice_global.yaml.j2 @@ -9,15 +9,15 @@ output geometry: variable change: variable change name: Soca2Cice arctic: - seaice edge: 0.8 - shuffle: false + seaice edge: 0.4 + shuffle: true rescale prior: rescale: true min hice: 0.5 min hsno: 0.1 antarctic: - seaice edge: 0.9 - shuffle: false # seg. fault when true + seaice edge: 0.4 + shuffle: true rescale prior: rescale: true min hice: 0.5 From a7361e93af3d106b2ce57c466cb963641a091dd7 Mon Sep 17 00:00:00 2001 From: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com> Date: Fri, 20 Dec 2024 10:39:17 -0500 Subject: [PATCH 075/199] update soil variables to be consistent with fv3-jedi changes (#56) When the GDASApp `sorc/fv3-jedi` hash is updated to [97e10d5](https://github.com/JCSDA-internal/fv3-jedi/commit/97e10d5addfe75debab7a6b5604e7a7443f00333) or a hash after this commit, it is necessary, at a minimum, to update soil variables in the following templates ``` model/atmosphere/atmosphere_background.yaml.j2 model/atmosphere/atmosphere_background_ensemble.yaml.j2 ``` This PR is opened to update the soil variables in the above templates. Resolves #49 **NOTE**: GDASApp PR [#1377](https://github.com/NOAA-EMC/GDASApp/pull/1377) requires this PR be merged into jcb-gdas `develop`. Once this is done, the `parm/jcb-gdas` hash in [#1377](https://github.com/NOAA-EMC/GDASApp/pull/1377) should be updated to point at the new jcb-gdas `develop`. --- parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 | 4 +--- .../model/atmosphere/atmosphere_background_ensemble.yaml.j2 | 4 +--- 2 files changed, 2 insertions(+), 6 deletions(-) diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 index 24a45f4b5..09a10c955 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 @@ -1,10 +1,8 @@ datapath: {{ atmosphere_background_path }} filetype: cube sphere history provider: ufs -ufs soil nlev: 4 -ufs fields split by level: [smc,stc] datetime: "{{ atmosphere_background_time_iso }}" filenames: [ {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_atmf006.nc, {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_sfcf006.nc ] state variables: [ua,va,t,delp,ps,sphum,ice_wat,liq_wat,o3mr,hgtsfc, - slmsk,sheleg,tmpsfc,vtype,stype,vfrac,stc,smc,snwdphMeters, + slmsk,sheleg,tmpsfc,vtype,stype,vfrac,soilt1,soilw1,snwdphMeters, u_srf,v_srf,f10m] diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_background_ensemble.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background_ensemble.yaml.j2 index 216788579..f3fed38e6 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_background_ensemble.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background_ensemble.yaml.j2 @@ -4,10 +4,8 @@ members from template: datetime: '{{ atmosphere_background_time_iso }}' filetype: cube sphere history provider: ufs - ufs soil nlev: 4 - ufs fields split by level: [smc,stc] state variables: [ua,va,t,delz,delp,ps,sphum,ice_wat,liq_wat,o3mr,hgtsfc, - slmsk,sheleg,tmpsfc,vtype,stype,vfrac,stc,smc,snwdphMeters, + slmsk,sheleg,tmpsfc,vtype,stype,vfrac,soilt1,soilw1,snwdphMeters, u_srf,v_srf,f10m] datapath: {{ atmosphere_background_ensemble_path }} filename is datetime templated: true From 86aff564dfd581ada0f831e71176b6e82bc9054c Mon Sep 17 00:00:00 2001 From: Jiarui Dong Date: Tue, 31 Dec 2024 12:35:24 -0500 Subject: [PATCH 076/199] Add diagnostic flags to the snow DA jcb yamls (#62) This PR adds the diagnostic flags to the jcb yamls for the in-situ snow observations. In addition, this PR updates the yamls by 1) re-organizing the order of the UFO filters and 2) adding several filters to synchronize with the offline workflow. --- .../jcb-gdas/observations/snow/sfcsno.yaml.j2 | 115 +++++++++++++++--- .../observations/snow/snocvr_snow.yaml.j2 | 108 +++++++++++++--- 2 files changed, 195 insertions(+), 28 deletions(-) diff --git a/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 b/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 index f1ef5d4e8..ce44417e9 100644 --- a/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 @@ -30,6 +30,30 @@ # Observation Filters (QC) # ------------------------ obs pre filters: + - filter: Create Diagnostic Flags + flags: + - name: missing_snowdepth + initial value: false + - name: missing_elevation + initial value: false + - name: temporal_thinning + initial value: false + - name: invalid_snowdepth + initial value: false + - name: invalid_elevation + initial value: false + - name: land_check + initial value: false + - name: landice_check + initial value: false + - name: elevation_bkgdiff + initial value: false + - name: rejectlist + initial value: false + - name: background_check + initial value: false + - name: buddy_check + initial value: false - filter: Perform Action filter variables: - name: totalSnowDepth @@ -44,53 +68,110 @@ - name: BkgError/totalSnowDepth_background_error type: float value: 30.0 + - filter: Domain Check + where: + - variable: + name: ObsValue/totalSnowDepth + value: is_valid + actions: + - name: set + flag: missing_snowdepth + ignore: rejected observations + - name: reject + - filter: Domain Check + where: + - variable: + name: MetaData/stationElevation + value: is_valid + actions: + - name: set + flag: missing_elevation + ignore: rejected observations + - name: reject + - filter: Temporal Thinning + min_spacing: '{{ window_length }}' + seed_time: '{{ snow_background_time_iso }}' + category_variable: + name: MetaData/stationIdentification + actions: + - name: set + flag: temporal_thinning + ignore: rejected observations + - name: reject obs prior filters: - filter: Bounds Check filter variables: - name: totalSnowDepth minvalue: 0.0 maxvalue: 20000.0 - action: - name: reject - - filter: Domain Check - where: - - variable: - name: MetaData/stationElevation - value: is_valid + actions: + - name: set + flag: invalid_snowdepth + ignore: rejected observations + - name: reject - filter: Domain Check # land only where: - variable: name: GeoVaLs/slmsk minvalue: 0.5 maxvalue: 1.5 + actions: + - name: set + flag: land_check + ignore: rejected observations + - name: reject + - filter: Domain Check + where: + - variable: + name: MetaData/stationElevation + minvalue: -200.0 + maxvalue: 9900.0 + actions: + - name: set + flag: invalid_elevation + ignore: rejected observations + - name: reject - filter: RejectList # no land-ice where: - variable: name: GeoVaLs/vtype minvalue: 14.5 maxvalue: 15.5 + actions: + - name: set + flag: landice_check + ignore: rejected observations + - name: reject - filter: Difference Check # elevation check reference: MetaData/stationElevation value: GeoVaLs/filtered_orography threshold: 200. + actions: + - name: set + flag: elevation_bkgdiff + ignore: rejected observations + - name: reject - filter: BlackList where: - variable: name: MetaData/stationIdentification is_in: [71120,71397,71621,71727,71816] size where true: 5 + actions: + - name: set + flag: rejectlist + ignore: rejected observations + - name: reject obs post filters: - filter: Background Check # gross error check filter variables: - name: totalSnowDepth threshold: 6.25 - action: - name: reject - - filter: Temporal Thinning - min_spacing: '{{ window_length }}' - seed_time: '{{ snow_background_time_iso }}' - category_variable: - name: MetaData/stationIdentification + actions: + - name: set + flag: background_check + ignore: rejected observations + - name: reject - filter: Met Office Buddy Check filter variables: - name: totalSnowDepth @@ -114,3 +195,9 @@ damping_factor_1: 1.0 damping_factor_2: 1.0 background_error_group: BkgError + actions: + - name: set + flag: buddy_check + ignore: rejected observations + - name: reject + diff --git a/parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 index 29ba1595e..fe73587b0 100644 --- a/parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 @@ -29,6 +29,28 @@ # Observation Filters (QC) # ------------------------ obs pre filters: + - filter: Create Diagnostic Flags + flags: + - name: missing_snowdepth + initial value: false + - name: missing_elevation + initial value: false + - name: temporal_thinning + initial value: false + - name: invalid_snowdepth + initial value: false + - name: invalid_elevation + initial value: false + - name: land_check + initial value: false + - name: landice_check + initial value: false + - name: elevation_bkgdiff + initial value: false + - name: background_check + initial value: false + - name: buddy_check + initial value: false - filter: Perform Action filter variables: - name: totalSnowDepth @@ -43,47 +65,99 @@ - name: BkgError/totalSnowDepth_background_error type: float value: 30.0 + - filter: Domain Check + where: + - variable: + name: ObsValue/totalSnowDepth + value: is_valid + actions: + - name: set + flag: missing_snowdepth + ignore: rejected observations + - name: reject + - filter: Domain Check + where: + - variable: + name: MetaData/stationElevation + value: is_valid + actions: + - name: set + flag: missing_elevation + ignore: rejected observations + - name: reject + - filter: Temporal Thinning + min_spacing: '{{ window_length }}' + seed_time: '{{ snow_background_time_iso }}' + category_variable: + name: MetaData/stationIdentification + actions: + - name: set + flag: temporal_thinning + ignore: rejected observations + - name: reject obs prior filters: - filter: Bounds Check filter variables: - name: totalSnowDepth minvalue: 0.0 maxvalue: 20000.0 - action: - name: reject - - filter: Domain Check - where: - - variable: - name: MetaData/stationElevation - value: is_valid + actions: + - name: set + flag: invalid_snowdepth + ignore: rejected observations + - name: reject - filter: Domain Check # land only where: - variable: name: GeoVaLs/slmsk minvalue: 0.5 maxvalue: 1.5 + actions: + - name: set + flag: land_check + ignore: rejected observations + - name: reject + - filter: Domain Check + where: + - variable: + name: MetaData/stationElevation + minvalue: -200.0 + maxvalue: 9900.0 + actions: + - name: set + flag: invalid_elevation + ignore: rejected observations + - name: reject - filter: RejectList # no land-ice where: - variable: name: GeoVaLs/vtype minvalue: 14.5 maxvalue: 15.5 + actions: + - name: set + flag: landice_check + ignore: rejected observations + - name: reject - filter: Difference Check # elevation check reference: MetaData/stationElevation value: GeoVaLs/filtered_orography threshold: 200. + actions: + - name: set + flag: elevation_bkgdiff + ignore: rejected observations + - name: reject obs post filters: - filter: Background Check # gross error check filter variables: - name: totalSnowDepth threshold: 6.25 - action: - name: reject - - filter: Temporal Thinning - min_spacing: '{{ window_length }}' - seed_time: '{{ snow_background_time_iso }}' - category_variable: - name: MetaData/stationIdentification + actions: + - name: set + flag: background_check + ignore: rejected observations + - name: reject - filter: Met Office Buddy Check filter variables: - name: totalSnowDepth @@ -107,3 +181,9 @@ damping_factor_1: 1.0 damping_factor_2: 1.0 background_error_group: BkgError + actions: + - name: set + flag: buddy_check + ignore: rejected observations + - name: reject + From 23b7d155aaa1a7d47ded8acd579bd711b183cc94 Mon Sep 17 00:00:00 2001 From: Yaping Wang <49168260+ypwang19@users.noreply.github.com> Date: Fri, 3 Jan 2025 08:52:52 -0500 Subject: [PATCH 077/199] Update YAMLs for aerosol VarBC (#61) This PR updates jcb yamls for aerosol variational bias correction. Will add the obs bias section to yamls in observations/aero/ later as more work is needed to select proper predictors. Companion PRs: - https://github.com/NOAA-EMC/GDASApp/pull/1426 - https://github.com/NOAA-EMC/global-workflow/pull/3189 Co-authored-by: ypwang19 Co-authored-by: Cory Martin --- parm/jcb-gdas/algorithm/aero/aero_bias_staging.yaml.j2 | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 parm/jcb-gdas/algorithm/aero/aero_bias_staging.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/aero/aero_bias_staging.yaml.j2 b/parm/jcb-gdas/algorithm/aero/aero_bias_staging.yaml.j2 new file mode 100644 index 000000000..3a3187ff6 --- /dev/null +++ b/parm/jcb-gdas/algorithm/aero/aero_bias_staging.yaml.j2 @@ -0,0 +1,10 @@ +mkdir: +- '{{aero_obsbiasout_path}}' +copy: + {% for observation_from_jcb in observations %} + {% if use_observer(observation_from_jcb) %} + {% if observation_from_jcb in bias_files %} + - ['{{aero_obsbiasroot_path}}/{{aero_obsbiasin_prefix}}{{bias_files[observation_from_jcb]}}', '{{aero_obsbiasin_path}}'] + {% endif %} + {% endif %} + {% endfor %} From 474425bb7a6840f3a0c847ff56bcca4ea88f8eb6 Mon Sep 17 00:00:00 2001 From: jianjunj Date: Thu, 16 Jan 2025 12:55:38 -0500 Subject: [PATCH 078/199] Add precipitable clouds in microwave all-sky assimilations (#63) Add rain, snow, and graupel clouds for CRTM inputs. Add a prior filter to zero clouds over non-water surfaces. Add an option of clouds in the Hydrometeo check. Add gross error check threshold for precipition impacted channels. Add cold-air-outbreak filters. Read cloud fraction from GeoVals. Render list of not-bias corrected channels from JCB tables. Add configurations for atms_n21. --- .../atmosphere/atms_n21.yaml | 43 ++ .../observations/atmosphere/amsua_n19.yaml.j2 | 142 +++- .../observations/atmosphere/atms_n20.yaml.j2 | 157 +++- .../observations/atmosphere/atms_n21.yaml.j2 | 686 ++++++++++++++++++ .../observations/atmosphere/atms_npp.yaml.j2 | 141 +++- 5 files changed, 1155 insertions(+), 14 deletions(-) create mode 100644 parm/jcb-gdas/observation_chronicle/atmosphere/atms_n21.yaml create mode 100644 parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/atms_n21.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/atms_n21.yaml new file mode 100644 index 000000000..3311e650a --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/atmosphere/atms_n21.yaml @@ -0,0 +1,43 @@ +# Instrument metadata +# ------------------- +commissioned: 2023-05-01T12:00:00 + +observer_type: satellite # Type of chronicle to use + +# Instrument initial configuration +# -------------------------------- +channel_variables: + simulated: min + active: min + biascorrtd: min + error: max + error_cld: max + ermax: max + var_b: min + var_pg: min + icld_det: min + icloud: min + iaerosol: min +channel_values: + 1: [ 1, 1, 1, 4.50, 20.00, 4.50, 10.00, 0.00, -2, 1, -1 ] + 2: [ 1, 1, 1, 4.50, 25.00, 4.50, 10.00, 0.00, -2, 1, -1 ] + 3: [ 1, 1, 1, 4.50, 12.00, 3.00, 10.00, 0.00, -2, 1, -1 ] + 4: [ 1, 1, 1, 2.50, 7.00, 3.00, 10.00, 0.00, -2, 1, -1 ] + 5: [ 1, 1, 1, 0.55, 3.50, 1.00, 10.00, 0.00, -2, 1, -1 ] + 6: [ 1, 1, 1, 0.30, 3.00, 1.00, 10.00, 0.00, -2, 1, -1 ] + 7: [ 1, 1, 1, 0.30, 0.80, 1.00, 10.00, 0.00, -2, 1, -1 ] + 8: [ 1, 1, 1, 0.40, 0.40, 1.00, 10.00, 0.00, -2, 1, -1 ] + 9: [ 1, 1, 1, 0.40, 0.40, 1.00, 10.00, 0.00, -2, 1, -1 ] + 10: [ 1, 1, 1, 0.40, 0.40, 1.00, 10.00, 0.00, -2, 1, -1 ] + 11: [ 1, 1, 1, 0.45, 0.45, 1.00, 10.00, 0.00, -2, 1, -1 ] + 12: [ 1, 1, 1, 0.45, 0.45, 1.00, 10.00, 0.00, -2, 1, -1 ] + 13: [ 1, 1, 1, 0.55, 0.55, 1.00, 10.00, 0.00, -2, 1, -1 ] + 14: [ 1, 1, 1, 0.80, 0.80, 2.00, 10.00, 0.00, -2, 1, -1 ] + 15: [ 1, 1, 0, 4.00, 4.00, 4.50, 10.00, 0.00, -2, 1, -1 ] + 16: [ 1, 1, 1, 4.00, 19.00, 4.50, 10.00, 0.00, -2, 1, -1 ] + 17: [ 1, 1, 1, 4.00, 30.00, 2.00, 10.00, 0.00, -2, 1, -1 ] + 18: [ 1, 1, 1, 3.50, 25.00, 2.00, 10.00, 0.00, -2, 1, -1 ] + 19: [ 1, 1, 1, 3.00, 16.50, 2.00, 10.00, 0.00, -2, 1, -1 ] + 20: [ 1, 1, 1, 3.00, 12.00, 2.00, 10.00, 0.00, -2, 1, -1 ] + 21: [ 1, 1, 1, 3.00, 9.00, 2.00, 10.00, 0.00, -2, 1, -1 ] + 22: [ 1, 1, 1, 3.00, 6.50, 2.00, 10.00, 0.00, -2, 1, -1 ] diff --git a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 index dba4162db..2cc30e3d6 100644 --- a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 @@ -21,9 +21,10 @@ # -------------------- obs operator: name: CRTM - Absorbers: [H2O, O3] - Clouds: [Water, Ice] - Cloud_Fraction: 1.0 + Absorbers: [H2O, O3, CO2] + Clouds: &{{observation_from_jcb}}_clouds [Water, Ice, Rain, Snow, Graupel] +# Cloud_Fraction: 1.0 + Cloud_Seeding: true obs options: Sensor_ID: &{{observation_from_jcb}}_sensor_id amsua_n19 EndianType: little_endian @@ -34,6 +35,8 @@ obs bias: input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" + variables without bc: [brightnessTemperature] + channels: {{ get_satellite_variable(observation_from_jcb, "not_biascorrtd") }} variational bc: predictors: - name: constant @@ -64,7 +67,25 @@ # Observation Filters (QC) # ------------------------ - obs filters: + obs prior filters: + # Zero Atmospheric clouds in CRTM where water_area_fraction < 0.99 + - filter: Variable Assignment + assignments: + - name: MetaData/zeroCloudInCRTM + type: int + function: + name: ObsFunction/Conditional + options: +# firstmatchingcase: false + defaultvalue: 0 # Will not zero clouds by default + cases: + - where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + value: 1 # Will zero clouds by default + + obs post filters: - filter: BlackList filter variables: - name: brightnessTemperature @@ -124,6 +145,7 @@ channels: *{{observation_from_jcb}}_simulated_channels options: channels: *{{observation_from_jcb}}_simulated_channels + Clouds: *{{observation_from_jcb}}_clouds obserr_clearsky: *{{observation_from_jcb}}_err0 clwret_function: name: ObsFunction/CLWRetMW @@ -301,6 +323,8 @@ x1: *{{observation_from_jcb}}_x1 err0: *{{observation_from_jcb}}_err0 err1: *{{observation_from_jcb}}_err1 + threshold: 3 + threshold_precip: 2.5 obserr_bound_max: [{{ get_satellite_variable(observation_from_jcb, "ermax") }}] action: name: reject @@ -334,8 +358,116 @@ minvalue: 1.0e-12 action: name: reject +# Column liquid cloud + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnLiquidCloud + type: float + function: + name: TotalColumnCloudGuess@ObsFunction + options: + cloud_content: mass_content_of_cloud_liquid_water_in_atmosphere_layer +# Column ice cloud + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnIceCloud + type: float + function: + name: TotalColumnCloudGuess@ObsFunction + options: + cloud_content: mass_content_of_cloud_ice_in_atmosphere_layer +# Combined ColumnLiquidCloud + ColumnIceCloud + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnLiquidIceCloud + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: DerivedMetaData/ColumnLiquidCloud + - name: DerivedMetaData/ColumnIceCloud + coefs: [1, 1] +# replace zero by 1e-7 in DerivedMetaData/ColumnLiquidIceCloud + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnLiquidIceCloud + type: float + value: 1.0e-7 + where: + - variable: + name: DerivedMetaData/ColumnLiquidIceCloud + maxvalue: 1.0e-8 - +# ratio of liquid /(Liquid+Ice) + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnRatioLiquidCloud + type: float + function: + name: ObsFunction/ElementMultiply + options: + variables: + - name: DerivedMetaData/ColumnLiquidCloud + - name: DerivedMetaData/ColumnLiquidIceCloud + exponents: [1,-1] +# TotalColumnVaporGuess + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/TotalColumnVaporGuess + type: float + function: + name: ObsFunction/TotalColumnVaporGuess +# potentialTemperature at surface + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureSurface + type: float + function: + name: PotentialTemperatureGuess2D@ObsFunction + options: + approximate pressure: surface_pressure +# potentialTemperature near 700 hPa + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureAt700hPa + type: float + function: + name: PotentialTemperatureGuess2D@ObsFunction + options: + approximate pressure: 70000.0 +# stability = Difference between PotentialTemperatur values + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureDiff + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: DerivedMetaData/PotentialTemperatureAt700hPa + - name: DerivedMetaData/PotentialTemperatureSurface + coefs: [1, -1] +## cao_check + - filter: BlackList + filter variables: + - name: brightnessTemperature + channels: 1-6, 15 + where: + - variable: + name: ObsFunction/TotalColumnVaporGuess + maxvalue: 8.0 + - variable: + name: DerivedMetaData/ColumnRatioLiquidCloud + maxvalue: 0.5 + - variable: + name: DerivedMetaData/ColumnLiquidIceCloud + minvalue: 1.0e-6 + - variable: + name: DerivedMetaData/PotentialTemperatureDiff + maxvalue: 12.0 + action: + name: reject # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: diff --git a/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 index c16a22c51..82d2f5037 100644 --- a/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 @@ -22,8 +22,8 @@ obs operator: name: CRTM Absorbers: [H2O, O3, CO2] - Clouds: [Water, Ice] - Cloud_Fraction: 1.0 + Clouds: &{{observation_from_jcb}}_clouds [Water, Ice, Rain, Snow, Graupel] +# Cloud_Fraction: 1.0 Cloud_Seeding: true obs options: Sensor_ID: &{{observation_from_jcb}}_sensor_id atms_n20 @@ -37,6 +37,8 @@ obs bias: input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" + variables without bc: [brightnessTemperature] + channels: {{ get_satellite_variable(observation_from_jcb, "not_biascorrtd") }} variational bc: predictors: - name: constant @@ -65,6 +67,25 @@ ratio for small dataset: 2.0 output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" + # ------------------------ + obs prior filters: + # Zero Atmospheric clouds in CRTM where water_area_fraction < 0.99 + - filter: Variable Assignment + assignments: + - name: MetaData/zeroCloudInCRTM + type: int + function: + name: ObsFunction/Conditional + options: +# firstmatchingcase: false + defaultvalue: 0 # Will not zero clouds by default + cases: + - where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + value: 1 # Will zero clouds by default + # Observation Pre Filters (QC) # ---------------------------- obs pre filters: @@ -114,6 +135,7 @@ clwret_ch238: 1 clwret_ch314: 2 clwret_types: [ObsValue] + # Calculate CLW retrieved from observation - filter: Variable Assignment assignments: @@ -125,6 +147,7 @@ clwret_ch238: 1 clwret_ch314: 2 clwret_types: [HofX] + # Calculate symmetric retrieved CLW - filter: Variable Assignment assignments: @@ -166,6 +189,7 @@ scatret_ch314: 2 scatret_ch890: 16 scatret_types: [ObsValue] + # Calculate CLW obs/bkg match index - filter: Variable Assignment assignments: @@ -184,6 +208,7 @@ clwret_clearsky: [0.030, 0.030, 0.030, 0.020, 0.030, 0.080, 0.150, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.020, 0.030, 0.030, 0.030, 0.030, 0.050, 0.100] + # Calculate symmetric observation error - filter: Variable Assignment assignments: @@ -209,6 +234,7 @@ err1: [20.000, 25.000, 12.000, 7.000, 3.500, 3.000, 0.800, 0.400, 0.400, 0.400, 0.450, 0.450, 0.550, 0.800, 4.000, 19.000, 30.000, 25.000, 16.500, 12.000, 9.000, 6.500] + # Calculate Innovation@DerivedMetaData - filter: Variable Assignment assignments: @@ -225,6 +251,7 @@ - name: brightnessTemperature@HofX channels: *{{observation_from_jcb}}_simulated_channels coefs: [1, -1] + # Step 0-C: Assign Initial All-Sky Observation Error - filter: Perform Action filter variables: @@ -235,6 +262,7 @@ error function: name: InitialObsError@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels + # Step 1: Remove Observations from the Edge of the Scan - filter: Domain Check filter variables: @@ -322,6 +350,7 @@ obserr_function: name: InitialObsError@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels + Clouds: *{{observation_from_jcb}}_clouds - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -348,6 +377,7 @@ options: sensor: *{{observation_from_jcb}}_sensor_id channels: *{{observation_from_jcb}}_simulated_channels + - filter: Perform Action filter variables: - name: brightnessTemperature @@ -357,6 +387,7 @@ inflation variable: name: ObsErrorFactorTopo@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels + # Step 7: Obs Error Inflation based on TOA Transmittancec Check - filter: Variable Assignment assignments: @@ -368,6 +399,7 @@ channels: *{{observation_from_jcb}}_simulated_channels options: channels: *{{observation_from_jcb}}_simulated_channels + - filter: Perform Action filter variables: - name: brightnessTemperature @@ -377,6 +409,7 @@ inflation variable: name: ObsErrorFactorTransmitTop@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels + # Step 8: Observation Error Inflation based on Surface Jacobian Check - filter: Variable Assignment assignments: @@ -391,6 +424,7 @@ channels: *{{observation_from_jcb}}_simulated_channels obserr_demisf: [0.010, 0.020, 0.015, 0.020, 0.200] obserr_dtempf: [0.500, 2.000, 1.000, 2.000, 4.500] + - filter: Perform Action filter variables: - name: brightnessTemperature @@ -400,6 +434,7 @@ inflation variable: name: ObsErrorFactorSurfJacobian@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels + # Step 9: Situation Dependent Check - filter: Variable Assignment assignments: @@ -427,6 +462,7 @@ obserr_clearsky: [4.500, 4.500, 4.500, 2.500, 0.550, 0.300, 0.300, 0.400, 0.400, 0.400, 0.450, 0.450, 0.550, 0.800, 4.000, 4.000, 4.000, 3.500, 3.000, 3.000, 3.000, 3.000] + - filter: Perform Action filter variables: - name: brightnessTemperature @@ -436,8 +472,9 @@ inflation variable: name: ObsErrorFactorSituDepend@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels + # Step 10: Gross check - # Remove data if abs(Obs-HofX) > absolute threhold + # Remove data if abs(Obs-HofX) > absolute threshold - filter: Variable Assignment assignments: - name: ObsErrorFactorLat@DerivedMetaData @@ -446,6 +483,7 @@ name: ObsErrorFactorLatRad@ObsFunction options: latitude_parameters: [25.0, 0.25, 0.04, 3.0] + - filter: Variable Assignment assignments: - name: ObsErrorBound@DerivedMetaData @@ -470,9 +508,11 @@ obserr_function: name: InitialObsError@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels - threhold: 3 + threshold: 3 + threshold_precip: 2.5 obserr_bound_max: [4.5, 4.5, 3.0, 3.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 2.0, 4.5, 4.5, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0] + - filter: Background Check filter variables: - name: brightnessTemperature @@ -526,7 +566,116 @@ ignore: rejected observations - name: reject +# Column liquid cloud + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnLiquidCloud + type: float + function: + name: TotalColumnCloudGuess@ObsFunction + options: + cloud_content: mass_content_of_cloud_liquid_water_in_atmosphere_layer +# Column ice cloud + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnIceCloud + type: float + function: + name: TotalColumnCloudGuess@ObsFunction + options: + cloud_content: mass_content_of_cloud_ice_in_atmosphere_layer +# Combined ColumnLiquidCloud + ColumnIceCloud + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnLiquidIceCloud + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: DerivedMetaData/ColumnLiquidCloud + - name: DerivedMetaData/ColumnIceCloud + coefs: [1, 1] +# replace zero by 1e-7 in DerivedMetaData/ColumnLiquidIceCloud + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnLiquidIceCloud + type: float + value: 1.0e-7 + where: + - variable: + name: DerivedMetaData/ColumnLiquidIceCloud + maxvalue: 1.0e-8 +# ratio of liquid /(Liquid+Ice) + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnRatioLiquidCloud + type: float + function: + name: ObsFunction/ElementMultiply + options: + variables: + - name: DerivedMetaData/ColumnLiquidCloud + - name: DerivedMetaData/ColumnLiquidIceCloud + exponents: [1,-1] +# TotalColumnVaporGuess + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/TotalColumnVaporGuess + type: float + function: + name: ObsFunction/TotalColumnVaporGuess +# potentialTemperature at surface + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureSurface + type: float + function: + name: PotentialTemperatureGuess2D@ObsFunction + options: + approximate pressure: surface_pressure +# potentialTemperature near 700 hPa + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureAt700hPa + type: float + function: + name: PotentialTemperatureGuess2D@ObsFunction + options: + approximate pressure: 70000.0 +# stability = Difference between PotentialTemperatur values + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureDiff + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: DerivedMetaData/PotentialTemperatureAt700hPa + - name: DerivedMetaData/PotentialTemperatureSurface + coefs: [1, -1] +## cao_check + - filter: BlackList + filter variables: + - name: brightnessTemperature + channels: 1-7, 16-22 + where: + - variable: + name: ObsFunction/TotalColumnVaporGuess + maxvalue: 8.0 + - variable: + name: DerivedMetaData/ColumnRatioLiquidCloud + maxvalue: 0.5 + - variable: + name: DerivedMetaData/ColumnLiquidIceCloud + minvalue: 1.0e-6 + - variable: + name: DerivedMetaData/PotentialTemperatureDiff + maxvalue: 12.0 + action: + name: reject # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: diff --git a/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 new file mode 100644 index 000000000..560be3b89 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 @@ -0,0 +1,686 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: ATMS N21 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [brightnessTemperature] + channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} + + # Observation Operator + # -------------------- + obs operator: + name: CRTM + Absorbers: [H2O, O3, CO2] + Clouds: &{{observation_from_jcb}}_clouds [Water, Ice, Rain, Snow, Graupel] +# Cloud_Fraction: 1.0 + Cloud_Seeding: true + obs options: + Sensor_ID: &{{observation_from_jcb}}_sensor_id atms_n21 + EndianType: little_endian + CoefficientPath: "{{crtm_coefficient_path}}" + linear obs operator: + Absorbers: [H2O, O3] + + # Observation Bias Correction (VarBC) + # ----------------------------------- + obs bias: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" + variables without bc: [brightnessTemperature] + channels: {{ get_satellite_variable(observation_from_jcb, "not_biascorrtd") }} + variational bc: + predictors: + - name: constant + - name: lapseRate + order: 2 + tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" + - name: lapseRate + tlapse: *{{observation_from_jcb}}_tlapse + - name: emissivityJacobian + - name: sensorScanAngle + order: 4 + - name: sensorScanAngle + order: 3 + - name: sensorScanAngle + order: 2 + - name: sensorScanAngle + covariance: + minimal required obs number: 20 + variance range: [1.0e-6, 10.0] + step size: 1.0e-4 + largest analysis variance: 10000.0 + prior: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiascovin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovin_suffix}}" + inflation: + ratio: 1.1 + ratio for small dataset: 2.0 + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" + + # ------------------------ + obs prior filters: + # Zero Atmospheric clouds in CRTM where water_area_fraction < 0.99 + - filter: Variable Assignment + assignments: + - name: MetaData/zeroCloudInCRTM + type: int + function: + name: ObsFunction/Conditional + options: +# firstmatchingcase: false + defaultvalue: 0 # Will not zero clouds by default + cases: + - where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + value: 1 # Will zero clouds by default + + # Observation Pre Filters (QC) + # ---------------------------- + obs pre filters: + # Step 0-A: Create Diagnostic Flags + - filter: Create Diagnostic Flags + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + flags: + - name: ScanEdgeRemoval + initial value: false + force reinitialization: false + - name: Thinning + initial value: false + force reinitialization: false + - name: CLWRetrievalCheck + initial value: false + force reinitialization: false + - name: WindowChannelExtremeResidual + initial value: false + force reinitialization: false + - name: HydrometeorCheck + initial value: false + force reinitialization: false + - name: GrossCheck + initial value: false + force reinitialization: false + - name: InterChannelConsistency + initial value: false + force reinitialization: false + - name: UseflagCheck + initial value: false + force reinitialization: false + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Step 0-B: Calculate derived variables + # Calculate CLW retrieved from observation + - filter: Variable Assignment + assignments: + - name: CLWRetFromObs@DerivedMetaData + type: float + function: + name: CLWRetMW@ObsFunction + options: + clwret_ch238: 1 + clwret_ch314: 2 + clwret_types: [ObsValue] + + # Calculate CLW retrieved from observation + - filter: Variable Assignment + assignments: + - name: CLWRetFromBkg@DerivedMetaData + type: float + function: + name: CLWRetMW@ObsFunction + options: + clwret_ch238: 1 + clwret_ch314: 2 + clwret_types: [HofX] + + # Calculate symmetric retrieved CLW + - filter: Variable Assignment + assignments: + - name: CLWRetSymmetric@DerivedMetaData + type: float + value: 1000.0 + + - filter: Variable Assignment + where: + - variable: + name: CLWRetFromObs@DerivedMetaData + minvalue: 0. + maxvalue: 999. + - variable: + name: CLWRetFromBkg@DerivedMetaData + minvalue: 0. + maxvalue: 999. + where operator: and + assignments: + - name: CLWRetSymmetric@DerivedMetaData + type: float + function: + name: Arithmetic@ObsFunction + options: + variables: + - name: CLWRetFromObs@DerivedMetaData + - name: CLWRetFromBkg@DerivedMetaData + total coefficient: 0.5 + + # Calculate scattering index from observation + - filter: Variable Assignment + assignments: + - name: SIRetFromObs@DerivedMetaData + type: float + function: + name: SCATRetMW@ObsFunction + options: + scatret_ch238: 1 + scatret_ch314: 2 + scatret_ch890: 16 + scatret_types: [ObsValue] + + # Calculate CLW obs/bkg match index + - filter: Variable Assignment + assignments: + - name: CLWMatchIndex@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: CLWMatchIndexMW@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + clwobs_function: + name: CLWRetFromObs@DerivedMetaData + clwbkg_function: + name: CLWRetFromBkg@DerivedMetaData + clwret_clearsky: [0.030, 0.030, 0.030, 0.020, 0.030, 0.080, 0.150, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.020, 0.030, 0.030, + 0.030, 0.030, 0.050, 0.100] + + # Calculate symmetric observation error + - filter: Variable Assignment + assignments: + - name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorModelRamp@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + xvar: + name: CLWRetSymmetric@DerivedMetaData + x0: [0.030, 0.030, 0.030, 0.020, 0.030, 0.080, 0.150, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.000, 0.020, 0.030, 0.030, 0.030, 0.030, + 0.050, 0.100] + x1: [0.350, 0.380, 0.400, 0.450, 0.500, 1.000, 1.000, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.000, 0.350, 0.500, 0.500, 0.500, 0.500, + 0.500, 0.500] + err0: [4.500, 4.500, 4.500, 2.500, 0.550, 0.300, 0.300, 0.400, 0.400, 0.400, + 0.450, 0.450, 0.550, 0.800, 4.000, 4.000, 4.000, 3.500, 3.000, 3.000, + 3.000, 3.000] + err1: [20.000, 25.000, 12.000, 7.000, 3.500, 3.000, 0.800, 0.400, 0.400, + 0.400, 0.450, 0.450, 0.550, 0.800, 4.000, 19.000, 30.000, 25.000, 16.500, + 12.000, 9.000, 6.500] + + # Calculate Innovation@DerivedMetaData + - filter: Variable Assignment + assignments: + - name: Innovation@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsFunction/Arithmetic + channels: *{{observation_from_jcb}}_simulated_channels + options: + variables: + - name: brightnessTemperature@ObsValue + channels: *{{observation_from_jcb}}_simulated_channels + - name: brightnessTemperature@HofX + channels: *{{observation_from_jcb}}_simulated_channels + coefs: [1, -1] + + # Step 0-C: Assign Initial All-Sky Observation Error + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: assign error + error function: + name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 1: Remove Observations from the Edge of the Scan + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: 1-22 + where: + - variable: + name: MetaData/sensorScanPosition + is_in: 7-90 + actions: + - name: set + flag: ScanEdgeRemoval + - name: reject + + # Step 2: Data Thinning + - filter: Gaussian Thinning + horizontal_mesh: 145 + use_reduced_horizontal_grid: true + distance_norm: geodesic + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + actions: + - name: set + flag: Thinning + - name: reject + + # Step 3A: CLW Retrieval Check (observation_based) + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: 1-7, 16-22 + test variables: + - name: CLWRetFromObs@DerivedMetaData + maxvalue: 999.0 + actions: + - name: set + flag: CLWRetrievalCheck + - name: reject + + # Step 3B: CLW Retrieval Check (background_based) + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: 1-7, 16-22 + test variables: + - name: CLWRetFromBkg@DerivedMetaData + maxvalue: 999.0 + actions: + - name: set + flag: CLWRetrievalCheck + - name: reject + + # Step 4: Window Channel Sanity Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: 1-7, 16, 17-22 + test variables: + - name: Innovation@DerivedMetaData + channels: 1, 2, 5-7, 16 + maxvalue: 200.0 + minvalue: -200.0 + flag all filter variables if any test variable is out of bounds: true + actions: + - name: set + flag: WindowChannelExtremeResidual + - name: reject + + # Step 5: Hydrometeor Check (cloud/precipitation affected chanels) + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/HydrometeorCheckATMS + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: HydrometeorCheckATMS@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_clearsky: [4.500, 4.500, 4.500, 2.500, 0.550, 0.300, 0.300, 0.400, + 0.400, 0.400, 0.450, 0.450, 0.550, 0.800, 4.000, 4.000, 4.000, 3.500, + 3.000, 3.000, 3.000, 3.000] + clwret_function: + name: CLWRetFromObs@DerivedMetaData + obserr_function: + name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + Clouds: *{{observation_from_jcb}}_clouds + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: DerivedMetaData/HydrometeorCheckATMS + channels: *{{observation_from_jcb}}_simulated_channels + maxvalue: 0.0 + actions: + - name: set + flag: HydrometeorCheck + ignore: rejected observations + - name: reject + + # Step 6: Observation Error Inflation based on Topography Check + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorTopo@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorFactorTopoRad@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsErrorFactorTopo@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 7: Obs Error Inflation based on TOA Transmittancec Check + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorTransmitTop@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorFactorTransmitTopRad@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsErrorFactorTransmitTop@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 8: Observation Error Inflation based on Surface Jacobian Check + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorSurfJacobian@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorFactorSurfJacobianRad@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + obserr_demisf: [0.010, 0.020, 0.015, 0.020, 0.200] + obserr_dtempf: [0.500, 2.000, 1.000, 2.000, 4.500] + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsErrorFactorSurfJacobian@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 9: Situation Dependent Check + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorSituDepend@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorFactorSituDependMW@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + clwbkg_function: + name: CLWRetFromBkg@DerivedMetaData + clwobs_function: + name: CLWRetFromObs@DerivedMetaData + scatobs_function: + name: SIRetFromObs@DerivedMetaData + clwmatchidx_function: + name: CLWMatchIndex@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + obserr_function: + name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + obserr_clearsky: [4.500, 4.500, 4.500, 2.500, 0.550, 0.300, 0.300, 0.400, + 0.400, 0.400, 0.450, 0.450, 0.550, 0.800, 4.000, 4.000, 4.000, 3.500, + 3.000, 3.000, 3.000, 3.000] + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsErrorFactorSituDepend@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 10: Gross check + # Remove data if abs(Obs-HofX) > absolute threshold + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorLat@DerivedMetaData + type: float + function: + name: ObsErrorFactorLatRad@ObsFunction + options: + latitude_parameters: [25.0, 0.25, 0.04, 3.0] + + - filter: Variable Assignment + assignments: + - name: ObsErrorBound@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorBoundMW@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_latitude: + name: ObsErrorFactorLat@DerivedMetaData + obserr_bound_transmittop: + name: ObsErrorFactorTransmitTop@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_topo: + name: ObsErrorFactorTopo@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + obserr_function: + name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + threshold: 3 + threshold_precip: 2.5 + obserr_bound_max: [4.5, 4.5, 3.0, 3.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, + 1.0, 1.0, 2.0, 4.5, 4.5, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0] + + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + function absolute threshold: + - name: ObsErrorBound@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + actions: + - name: set + flag: GrossCheck + ignore: rejected observations + - name: reject + + # Step 11: Inter-Channel Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: InterChannelConsistencyCheck@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use passive_bc: true + sensor: *{{observation_from_jcb}}_sensor_id + use_flag: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 4, 1, 1, 1, 1, 1, 1, + 1] + maxvalue: 1.0e-12 + actions: + - name: set + flag: InterChannelConsistency + ignore: rejected observations + - name: reject + + # Step 12: Useflag Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/ChannelUseflagCheckRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 4, 1, 1, 1, 1, 1, 1, + 1] + minvalue: 1.0e-12 + actions: + - name: set + flag: UseflagCheck + ignore: rejected observations + - name: reject + +# Column liquid cloud + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnLiquidCloud + type: float + function: + name: TotalColumnCloudGuess@ObsFunction + options: + cloud_content: mass_content_of_cloud_liquid_water_in_atmosphere_layer +# Column ice cloud + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnIceCloud + type: float + function: + name: TotalColumnCloudGuess@ObsFunction + options: + cloud_content: mass_content_of_cloud_ice_in_atmosphere_layer +# Combined ColumnLiquidCloud + ColumnIceCloud + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnLiquidIceCloud + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: DerivedMetaData/ColumnLiquidCloud + - name: DerivedMetaData/ColumnIceCloud + coefs: [1, 1] +# replace zero by 1e-7 in DerivedMetaData/ColumnLiquidIceCloud + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnLiquidIceCloud + type: float + value: 1.0e-7 + where: + - variable: + name: DerivedMetaData/ColumnLiquidIceCloud + maxvalue: 1.0e-8 + +# ratio of liquid /(Liquid+Ice) + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnRatioLiquidCloud + type: float + function: + name: ObsFunction/ElementMultiply + options: + variables: + - name: DerivedMetaData/ColumnLiquidCloud + - name: DerivedMetaData/ColumnLiquidIceCloud + exponents: [1,-1] +# TotalColumnVaporGuess + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/TotalColumnVaporGuess + type: float + function: + name: ObsFunction/TotalColumnVaporGuess +# potentialTemperature at surface + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureSurface + type: float + function: + name: PotentialTemperatureGuess2D@ObsFunction + options: + approximate pressure: surface_pressure +# potentialTemperature near 700 hPa + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureAt700hPa + type: float + function: + name: PotentialTemperatureGuess2D@ObsFunction + options: + approximate pressure: 70000.0 +# stability = Difference between PotentialTemperatur values + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureDiff + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: DerivedMetaData/PotentialTemperatureAt700hPa + - name: DerivedMetaData/PotentialTemperatureSurface + coefs: [1, -1] +## cao_check + - filter: BlackList + filter variables: + - name: brightnessTemperature + channels: 1-7, 16-22 + where: + - variable: + name: ObsFunction/TotalColumnVaporGuess + maxvalue: 8.0 + - variable: + name: DerivedMetaData/ColumnRatioLiquidCloud + maxvalue: 0.5 + - variable: + name: DerivedMetaData/ColumnLiquidIceCloud + minvalue: 1.0e-6 + - variable: + name: DerivedMetaData/PotentialTemperatureDiff + maxvalue: 12.0 + action: + name: reject + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 index d18985cd8..309da3466 100644 --- a/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 @@ -22,8 +22,8 @@ obs operator: name: CRTM Absorbers: [H2O, O3, CO2] - Clouds: [Water, Ice] - Cloud_Fraction: 1.0 + Clouds: &{{observation_from_jcb}}_clouds [Water, Ice, Rain, Snow, Graupel] +# Cloud_Fraction: 1.0 Cloud_Seeding: true obs options: Sensor_ID: &{{observation_from_jcb}}_sensor_id atms_npp @@ -37,6 +37,8 @@ obs bias: input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" + variables without bc: [brightnessTemperature] + channels: {{ get_satellite_variable(observation_from_jcb, "not_biascorrtd") }} variational bc: predictors: - name: constant @@ -65,6 +67,25 @@ ratio for small dataset: 2.0 output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" + # ------------------------ + obs prior filters: + # Zero Atmospheric clouds in CRTM where water_area_fraction < 0.99 + - filter: Variable Assignment + assignments: + - name: MetaData/zeroCloudInCRTM + type: int + function: + name: ObsFunction/Conditional + options: +# firstmatchingcase: false + defaultvalue: 0 # Will not zero clouds by default + cases: + - where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + value: 1 # Will zero clouds by default + # Observation Pre Filters (QC) # ---------------------------- obs pre filters: @@ -329,7 +350,7 @@ obserr_function: name: InitialObsError@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels - + Clouds: *{{observation_from_jcb}}_clouds - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -453,7 +474,7 @@ channels: *{{observation_from_jcb}}_simulated_channels # Step 10: Gross check - # Remove data if abs(Obs-HofX) > absolute threhold + # Remove data if abs(Obs-HofX) > absolute threshold - filter: Variable Assignment assignments: - name: ObsErrorFactorLat@DerivedMetaData @@ -487,7 +508,8 @@ obserr_function: name: InitialObsError@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels - threhold: 3 + threshold: 3 + threshold_precip: 2.5 obserr_bound_max: [4.5, 4.5, 3.0, 3.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 2.0, 4.5, 4.5, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0] @@ -544,7 +566,116 @@ ignore: rejected observations - name: reject +# Column liquid cloud + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnLiquidCloud + type: float + function: + name: TotalColumnCloudGuess@ObsFunction + options: + cloud_content: mass_content_of_cloud_liquid_water_in_atmosphere_layer +# Column ice cloud + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnIceCloud + type: float + function: + name: TotalColumnCloudGuess@ObsFunction + options: + cloud_content: mass_content_of_cloud_ice_in_atmosphere_layer +# Combined ColumnLiquidCloud + ColumnIceCloud + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnLiquidIceCloud + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: DerivedMetaData/ColumnLiquidCloud + - name: DerivedMetaData/ColumnIceCloud + coefs: [1, 1] +# replace zero by 1e-7 in DerivedMetaData/ColumnLiquidIceCloud + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnLiquidIceCloud + type: float + value: 1.0e-7 + where: + - variable: + name: DerivedMetaData/ColumnLiquidIceCloud + maxvalue: 1.0e-8 +# ratio of liquid /(Liquid+Ice) + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnRatioLiquidCloud + type: float + function: + name: ObsFunction/ElementMultiply + options: + variables: + - name: DerivedMetaData/ColumnLiquidCloud + - name: DerivedMetaData/ColumnLiquidIceCloud + exponents: [1,-1] +# TotalColumnVaporGuess + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/TotalColumnVaporGuess + type: float + function: + name: ObsFunction/TotalColumnVaporGuess +# potentialTemperature at surface + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureSurface + type: float + function: + name: PotentialTemperatureGuess2D@ObsFunction + options: + approximate pressure: surface_pressure +# potentialTemperature near 700 hPa + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureAt700hPa + type: float + function: + name: PotentialTemperatureGuess2D@ObsFunction + options: + approximate pressure: 70000.0 +# stability = Difference between PotentialTemperatur values + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureDiff + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: DerivedMetaData/PotentialTemperatureAt700hPa + - name: DerivedMetaData/PotentialTemperatureSurface + coefs: [1, -1] +## cao_check + - filter: BlackList + filter variables: + - name: brightnessTemperature + channels: 1-7, 16-22 + where: + - variable: + name: ObsFunction/TotalColumnVaporGuess + maxvalue: 8.0 + - variable: + name: DerivedMetaData/ColumnRatioLiquidCloud + maxvalue: 0.5 + - variable: + name: DerivedMetaData/ColumnLiquidIceCloud + minvalue: 1.0e-6 + - variable: + name: DerivedMetaData/PotentialTemperatureDiff + maxvalue: 12.0 + action: + name: reject # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: From f1bc836e23dc91984e96c69062acece7b28d6426 Mon Sep 17 00:00:00 2001 From: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com> Date: Fri, 17 Jan 2025 21:41:42 -0500 Subject: [PATCH 079/199] Revert " Add precipitable clouds in microwave all-sky assimilations" (#68) --- .../atmosphere/atms_n21.yaml | 43 -- .../observations/atmosphere/amsua_n19.yaml.j2 | 142 +--- .../observations/atmosphere/atms_n20.yaml.j2 | 157 +--- .../observations/atmosphere/atms_n21.yaml.j2 | 686 ------------------ .../observations/atmosphere/atms_npp.yaml.j2 | 141 +--- 5 files changed, 14 insertions(+), 1155 deletions(-) delete mode 100644 parm/jcb-gdas/observation_chronicle/atmosphere/atms_n21.yaml delete mode 100644 parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/atms_n21.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/atms_n21.yaml deleted file mode 100644 index 3311e650a..000000000 --- a/parm/jcb-gdas/observation_chronicle/atmosphere/atms_n21.yaml +++ /dev/null @@ -1,43 +0,0 @@ -# Instrument metadata -# ------------------- -commissioned: 2023-05-01T12:00:00 - -observer_type: satellite # Type of chronicle to use - -# Instrument initial configuration -# -------------------------------- -channel_variables: - simulated: min - active: min - biascorrtd: min - error: max - error_cld: max - ermax: max - var_b: min - var_pg: min - icld_det: min - icloud: min - iaerosol: min -channel_values: - 1: [ 1, 1, 1, 4.50, 20.00, 4.50, 10.00, 0.00, -2, 1, -1 ] - 2: [ 1, 1, 1, 4.50, 25.00, 4.50, 10.00, 0.00, -2, 1, -1 ] - 3: [ 1, 1, 1, 4.50, 12.00, 3.00, 10.00, 0.00, -2, 1, -1 ] - 4: [ 1, 1, 1, 2.50, 7.00, 3.00, 10.00, 0.00, -2, 1, -1 ] - 5: [ 1, 1, 1, 0.55, 3.50, 1.00, 10.00, 0.00, -2, 1, -1 ] - 6: [ 1, 1, 1, 0.30, 3.00, 1.00, 10.00, 0.00, -2, 1, -1 ] - 7: [ 1, 1, 1, 0.30, 0.80, 1.00, 10.00, 0.00, -2, 1, -1 ] - 8: [ 1, 1, 1, 0.40, 0.40, 1.00, 10.00, 0.00, -2, 1, -1 ] - 9: [ 1, 1, 1, 0.40, 0.40, 1.00, 10.00, 0.00, -2, 1, -1 ] - 10: [ 1, 1, 1, 0.40, 0.40, 1.00, 10.00, 0.00, -2, 1, -1 ] - 11: [ 1, 1, 1, 0.45, 0.45, 1.00, 10.00, 0.00, -2, 1, -1 ] - 12: [ 1, 1, 1, 0.45, 0.45, 1.00, 10.00, 0.00, -2, 1, -1 ] - 13: [ 1, 1, 1, 0.55, 0.55, 1.00, 10.00, 0.00, -2, 1, -1 ] - 14: [ 1, 1, 1, 0.80, 0.80, 2.00, 10.00, 0.00, -2, 1, -1 ] - 15: [ 1, 1, 0, 4.00, 4.00, 4.50, 10.00, 0.00, -2, 1, -1 ] - 16: [ 1, 1, 1, 4.00, 19.00, 4.50, 10.00, 0.00, -2, 1, -1 ] - 17: [ 1, 1, 1, 4.00, 30.00, 2.00, 10.00, 0.00, -2, 1, -1 ] - 18: [ 1, 1, 1, 3.50, 25.00, 2.00, 10.00, 0.00, -2, 1, -1 ] - 19: [ 1, 1, 1, 3.00, 16.50, 2.00, 10.00, 0.00, -2, 1, -1 ] - 20: [ 1, 1, 1, 3.00, 12.00, 2.00, 10.00, 0.00, -2, 1, -1 ] - 21: [ 1, 1, 1, 3.00, 9.00, 2.00, 10.00, 0.00, -2, 1, -1 ] - 22: [ 1, 1, 1, 3.00, 6.50, 2.00, 10.00, 0.00, -2, 1, -1 ] diff --git a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 index 2cc30e3d6..dba4162db 100644 --- a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 @@ -21,10 +21,9 @@ # -------------------- obs operator: name: CRTM - Absorbers: [H2O, O3, CO2] - Clouds: &{{observation_from_jcb}}_clouds [Water, Ice, Rain, Snow, Graupel] -# Cloud_Fraction: 1.0 - Cloud_Seeding: true + Absorbers: [H2O, O3] + Clouds: [Water, Ice] + Cloud_Fraction: 1.0 obs options: Sensor_ID: &{{observation_from_jcb}}_sensor_id amsua_n19 EndianType: little_endian @@ -35,8 +34,6 @@ obs bias: input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" - variables without bc: [brightnessTemperature] - channels: {{ get_satellite_variable(observation_from_jcb, "not_biascorrtd") }} variational bc: predictors: - name: constant @@ -67,25 +64,7 @@ # Observation Filters (QC) # ------------------------ - obs prior filters: - # Zero Atmospheric clouds in CRTM where water_area_fraction < 0.99 - - filter: Variable Assignment - assignments: - - name: MetaData/zeroCloudInCRTM - type: int - function: - name: ObsFunction/Conditional - options: -# firstmatchingcase: false - defaultvalue: 0 # Will not zero clouds by default - cases: - - where: - - variable: - name: GeoVaLs/water_area_fraction - maxvalue: 0.99 - value: 1 # Will zero clouds by default - - obs post filters: + obs filters: - filter: BlackList filter variables: - name: brightnessTemperature @@ -145,7 +124,6 @@ channels: *{{observation_from_jcb}}_simulated_channels options: channels: *{{observation_from_jcb}}_simulated_channels - Clouds: *{{observation_from_jcb}}_clouds obserr_clearsky: *{{observation_from_jcb}}_err0 clwret_function: name: ObsFunction/CLWRetMW @@ -323,8 +301,6 @@ x1: *{{observation_from_jcb}}_x1 err0: *{{observation_from_jcb}}_err0 err1: *{{observation_from_jcb}}_err1 - threshold: 3 - threshold_precip: 2.5 obserr_bound_max: [{{ get_satellite_variable(observation_from_jcb, "ermax") }}] action: name: reject @@ -358,116 +334,8 @@ minvalue: 1.0e-12 action: name: reject -# Column liquid cloud - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/ColumnLiquidCloud - type: float - function: - name: TotalColumnCloudGuess@ObsFunction - options: - cloud_content: mass_content_of_cloud_liquid_water_in_atmosphere_layer -# Column ice cloud - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/ColumnIceCloud - type: float - function: - name: TotalColumnCloudGuess@ObsFunction - options: - cloud_content: mass_content_of_cloud_ice_in_atmosphere_layer -# Combined ColumnLiquidCloud + ColumnIceCloud - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/ColumnLiquidIceCloud - type: float - function: - name: ObsFunction/Arithmetic - options: - variables: - - name: DerivedMetaData/ColumnLiquidCloud - - name: DerivedMetaData/ColumnIceCloud - coefs: [1, 1] -# replace zero by 1e-7 in DerivedMetaData/ColumnLiquidIceCloud - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/ColumnLiquidIceCloud - type: float - value: 1.0e-7 - where: - - variable: - name: DerivedMetaData/ColumnLiquidIceCloud - maxvalue: 1.0e-8 -# ratio of liquid /(Liquid+Ice) - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/ColumnRatioLiquidCloud - type: float - function: - name: ObsFunction/ElementMultiply - options: - variables: - - name: DerivedMetaData/ColumnLiquidCloud - - name: DerivedMetaData/ColumnLiquidIceCloud - exponents: [1,-1] -# TotalColumnVaporGuess - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/TotalColumnVaporGuess - type: float - function: - name: ObsFunction/TotalColumnVaporGuess -# potentialTemperature at surface - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/PotentialTemperatureSurface - type: float - function: - name: PotentialTemperatureGuess2D@ObsFunction - options: - approximate pressure: surface_pressure -# potentialTemperature near 700 hPa - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/PotentialTemperatureAt700hPa - type: float - function: - name: PotentialTemperatureGuess2D@ObsFunction - options: - approximate pressure: 70000.0 -# stability = Difference between PotentialTemperatur values - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/PotentialTemperatureDiff - type: float - function: - name: ObsFunction/Arithmetic - options: - variables: - - name: DerivedMetaData/PotentialTemperatureAt700hPa - - name: DerivedMetaData/PotentialTemperatureSurface - coefs: [1, -1] -## cao_check - - filter: BlackList - filter variables: - - name: brightnessTemperature - channels: 1-6, 15 - where: - - variable: - name: ObsFunction/TotalColumnVaporGuess - maxvalue: 8.0 - - variable: - name: DerivedMetaData/ColumnRatioLiquidCloud - maxvalue: 0.5 - - variable: - name: DerivedMetaData/ColumnLiquidIceCloud - minvalue: 1.0e-6 - - variable: - name: DerivedMetaData/PotentialTemperatureDiff - maxvalue: 12.0 - action: - name: reject + # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: diff --git a/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 index 82d2f5037..c16a22c51 100644 --- a/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 @@ -22,8 +22,8 @@ obs operator: name: CRTM Absorbers: [H2O, O3, CO2] - Clouds: &{{observation_from_jcb}}_clouds [Water, Ice, Rain, Snow, Graupel] -# Cloud_Fraction: 1.0 + Clouds: [Water, Ice] + Cloud_Fraction: 1.0 Cloud_Seeding: true obs options: Sensor_ID: &{{observation_from_jcb}}_sensor_id atms_n20 @@ -37,8 +37,6 @@ obs bias: input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" - variables without bc: [brightnessTemperature] - channels: {{ get_satellite_variable(observation_from_jcb, "not_biascorrtd") }} variational bc: predictors: - name: constant @@ -67,25 +65,6 @@ ratio for small dataset: 2.0 output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" - # ------------------------ - obs prior filters: - # Zero Atmospheric clouds in CRTM where water_area_fraction < 0.99 - - filter: Variable Assignment - assignments: - - name: MetaData/zeroCloudInCRTM - type: int - function: - name: ObsFunction/Conditional - options: -# firstmatchingcase: false - defaultvalue: 0 # Will not zero clouds by default - cases: - - where: - - variable: - name: GeoVaLs/water_area_fraction - maxvalue: 0.99 - value: 1 # Will zero clouds by default - # Observation Pre Filters (QC) # ---------------------------- obs pre filters: @@ -135,7 +114,6 @@ clwret_ch238: 1 clwret_ch314: 2 clwret_types: [ObsValue] - # Calculate CLW retrieved from observation - filter: Variable Assignment assignments: @@ -147,7 +125,6 @@ clwret_ch238: 1 clwret_ch314: 2 clwret_types: [HofX] - # Calculate symmetric retrieved CLW - filter: Variable Assignment assignments: @@ -189,7 +166,6 @@ scatret_ch314: 2 scatret_ch890: 16 scatret_types: [ObsValue] - # Calculate CLW obs/bkg match index - filter: Variable Assignment assignments: @@ -208,7 +184,6 @@ clwret_clearsky: [0.030, 0.030, 0.030, 0.020, 0.030, 0.080, 0.150, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.020, 0.030, 0.030, 0.030, 0.030, 0.050, 0.100] - # Calculate symmetric observation error - filter: Variable Assignment assignments: @@ -234,7 +209,6 @@ err1: [20.000, 25.000, 12.000, 7.000, 3.500, 3.000, 0.800, 0.400, 0.400, 0.400, 0.450, 0.450, 0.550, 0.800, 4.000, 19.000, 30.000, 25.000, 16.500, 12.000, 9.000, 6.500] - # Calculate Innovation@DerivedMetaData - filter: Variable Assignment assignments: @@ -251,7 +225,6 @@ - name: brightnessTemperature@HofX channels: *{{observation_from_jcb}}_simulated_channels coefs: [1, -1] - # Step 0-C: Assign Initial All-Sky Observation Error - filter: Perform Action filter variables: @@ -262,7 +235,6 @@ error function: name: InitialObsError@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels - # Step 1: Remove Observations from the Edge of the Scan - filter: Domain Check filter variables: @@ -350,7 +322,6 @@ obserr_function: name: InitialObsError@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels - Clouds: *{{observation_from_jcb}}_clouds - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -377,7 +348,6 @@ options: sensor: *{{observation_from_jcb}}_sensor_id channels: *{{observation_from_jcb}}_simulated_channels - - filter: Perform Action filter variables: - name: brightnessTemperature @@ -387,7 +357,6 @@ inflation variable: name: ObsErrorFactorTopo@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels - # Step 7: Obs Error Inflation based on TOA Transmittancec Check - filter: Variable Assignment assignments: @@ -399,7 +368,6 @@ channels: *{{observation_from_jcb}}_simulated_channels options: channels: *{{observation_from_jcb}}_simulated_channels - - filter: Perform Action filter variables: - name: brightnessTemperature @@ -409,7 +377,6 @@ inflation variable: name: ObsErrorFactorTransmitTop@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels - # Step 8: Observation Error Inflation based on Surface Jacobian Check - filter: Variable Assignment assignments: @@ -424,7 +391,6 @@ channels: *{{observation_from_jcb}}_simulated_channels obserr_demisf: [0.010, 0.020, 0.015, 0.020, 0.200] obserr_dtempf: [0.500, 2.000, 1.000, 2.000, 4.500] - - filter: Perform Action filter variables: - name: brightnessTemperature @@ -434,7 +400,6 @@ inflation variable: name: ObsErrorFactorSurfJacobian@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels - # Step 9: Situation Dependent Check - filter: Variable Assignment assignments: @@ -462,7 +427,6 @@ obserr_clearsky: [4.500, 4.500, 4.500, 2.500, 0.550, 0.300, 0.300, 0.400, 0.400, 0.400, 0.450, 0.450, 0.550, 0.800, 4.000, 4.000, 4.000, 3.500, 3.000, 3.000, 3.000, 3.000] - - filter: Perform Action filter variables: - name: brightnessTemperature @@ -472,9 +436,8 @@ inflation variable: name: ObsErrorFactorSituDepend@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels - # Step 10: Gross check - # Remove data if abs(Obs-HofX) > absolute threshold + # Remove data if abs(Obs-HofX) > absolute threhold - filter: Variable Assignment assignments: - name: ObsErrorFactorLat@DerivedMetaData @@ -483,7 +446,6 @@ name: ObsErrorFactorLatRad@ObsFunction options: latitude_parameters: [25.0, 0.25, 0.04, 3.0] - - filter: Variable Assignment assignments: - name: ObsErrorBound@DerivedMetaData @@ -508,11 +470,9 @@ obserr_function: name: InitialObsError@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels - threshold: 3 - threshold_precip: 2.5 + threhold: 3 obserr_bound_max: [4.5, 4.5, 3.0, 3.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 2.0, 4.5, 4.5, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0] - - filter: Background Check filter variables: - name: brightnessTemperature @@ -566,116 +526,7 @@ ignore: rejected observations - name: reject -# Column liquid cloud - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/ColumnLiquidCloud - type: float - function: - name: TotalColumnCloudGuess@ObsFunction - options: - cloud_content: mass_content_of_cloud_liquid_water_in_atmosphere_layer -# Column ice cloud - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/ColumnIceCloud - type: float - function: - name: TotalColumnCloudGuess@ObsFunction - options: - cloud_content: mass_content_of_cloud_ice_in_atmosphere_layer -# Combined ColumnLiquidCloud + ColumnIceCloud - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/ColumnLiquidIceCloud - type: float - function: - name: ObsFunction/Arithmetic - options: - variables: - - name: DerivedMetaData/ColumnLiquidCloud - - name: DerivedMetaData/ColumnIceCloud - coefs: [1, 1] -# replace zero by 1e-7 in DerivedMetaData/ColumnLiquidIceCloud - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/ColumnLiquidIceCloud - type: float - value: 1.0e-7 - where: - - variable: - name: DerivedMetaData/ColumnLiquidIceCloud - maxvalue: 1.0e-8 -# ratio of liquid /(Liquid+Ice) - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/ColumnRatioLiquidCloud - type: float - function: - name: ObsFunction/ElementMultiply - options: - variables: - - name: DerivedMetaData/ColumnLiquidCloud - - name: DerivedMetaData/ColumnLiquidIceCloud - exponents: [1,-1] -# TotalColumnVaporGuess - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/TotalColumnVaporGuess - type: float - function: - name: ObsFunction/TotalColumnVaporGuess -# potentialTemperature at surface - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/PotentialTemperatureSurface - type: float - function: - name: PotentialTemperatureGuess2D@ObsFunction - options: - approximate pressure: surface_pressure -# potentialTemperature near 700 hPa - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/PotentialTemperatureAt700hPa - type: float - function: - name: PotentialTemperatureGuess2D@ObsFunction - options: - approximate pressure: 70000.0 -# stability = Difference between PotentialTemperatur values - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/PotentialTemperatureDiff - type: float - function: - name: ObsFunction/Arithmetic - options: - variables: - - name: DerivedMetaData/PotentialTemperatureAt700hPa - - name: DerivedMetaData/PotentialTemperatureSurface - coefs: [1, -1] -## cao_check - - filter: BlackList - filter variables: - - name: brightnessTemperature - channels: 1-7, 16-22 - where: - - variable: - name: ObsFunction/TotalColumnVaporGuess - maxvalue: 8.0 - - variable: - name: DerivedMetaData/ColumnRatioLiquidCloud - maxvalue: 0.5 - - variable: - name: DerivedMetaData/ColumnLiquidIceCloud - minvalue: 1.0e-6 - - variable: - name: DerivedMetaData/PotentialTemperatureDiff - maxvalue: 12.0 - action: - name: reject # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: diff --git a/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 deleted file mode 100644 index 560be3b89..000000000 --- a/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 +++ /dev/null @@ -1,686 +0,0 @@ -- - - # Observation Space (I/O) - # ----------------------- - obs space: - name: ATMS N21 - obsdatain: - engine: - type: H5File - obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" - obsdataout: - engine: - type: H5File - obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" - io pool: - max pool size: 1 - simulated variables: [brightnessTemperature] - channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} - - # Observation Operator - # -------------------- - obs operator: - name: CRTM - Absorbers: [H2O, O3, CO2] - Clouds: &{{observation_from_jcb}}_clouds [Water, Ice, Rain, Snow, Graupel] -# Cloud_Fraction: 1.0 - Cloud_Seeding: true - obs options: - Sensor_ID: &{{observation_from_jcb}}_sensor_id atms_n21 - EndianType: little_endian - CoefficientPath: "{{crtm_coefficient_path}}" - linear obs operator: - Absorbers: [H2O, O3] - - # Observation Bias Correction (VarBC) - # ----------------------------------- - obs bias: - input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" - output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" - variables without bc: [brightnessTemperature] - channels: {{ get_satellite_variable(observation_from_jcb, "not_biascorrtd") }} - variational bc: - predictors: - - name: constant - - name: lapseRate - order: 2 - tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" - - name: lapseRate - tlapse: *{{observation_from_jcb}}_tlapse - - name: emissivityJacobian - - name: sensorScanAngle - order: 4 - - name: sensorScanAngle - order: 3 - - name: sensorScanAngle - order: 2 - - name: sensorScanAngle - covariance: - minimal required obs number: 20 - variance range: [1.0e-6, 10.0] - step size: 1.0e-4 - largest analysis variance: 10000.0 - prior: - input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiascovin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovin_suffix}}" - inflation: - ratio: 1.1 - ratio for small dataset: 2.0 - output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" - - # ------------------------ - obs prior filters: - # Zero Atmospheric clouds in CRTM where water_area_fraction < 0.99 - - filter: Variable Assignment - assignments: - - name: MetaData/zeroCloudInCRTM - type: int - function: - name: ObsFunction/Conditional - options: -# firstmatchingcase: false - defaultvalue: 0 # Will not zero clouds by default - cases: - - where: - - variable: - name: GeoVaLs/water_area_fraction - maxvalue: 0.99 - value: 1 # Will zero clouds by default - - # Observation Pre Filters (QC) - # ---------------------------- - obs pre filters: - # Step 0-A: Create Diagnostic Flags - - filter: Create Diagnostic Flags - filter variables: - - name: brightnessTemperature - channels: *{{observation_from_jcb}}_simulated_channels - flags: - - name: ScanEdgeRemoval - initial value: false - force reinitialization: false - - name: Thinning - initial value: false - force reinitialization: false - - name: CLWRetrievalCheck - initial value: false - force reinitialization: false - - name: WindowChannelExtremeResidual - initial value: false - force reinitialization: false - - name: HydrometeorCheck - initial value: false - force reinitialization: false - - name: GrossCheck - initial value: false - force reinitialization: false - - name: InterChannelConsistency - initial value: false - force reinitialization: false - - name: UseflagCheck - initial value: false - force reinitialization: false - - # Observation Post Filters (QC) - # ----------------------------- - obs post filters: - # Step 0-B: Calculate derived variables - # Calculate CLW retrieved from observation - - filter: Variable Assignment - assignments: - - name: CLWRetFromObs@DerivedMetaData - type: float - function: - name: CLWRetMW@ObsFunction - options: - clwret_ch238: 1 - clwret_ch314: 2 - clwret_types: [ObsValue] - - # Calculate CLW retrieved from observation - - filter: Variable Assignment - assignments: - - name: CLWRetFromBkg@DerivedMetaData - type: float - function: - name: CLWRetMW@ObsFunction - options: - clwret_ch238: 1 - clwret_ch314: 2 - clwret_types: [HofX] - - # Calculate symmetric retrieved CLW - - filter: Variable Assignment - assignments: - - name: CLWRetSymmetric@DerivedMetaData - type: float - value: 1000.0 - - - filter: Variable Assignment - where: - - variable: - name: CLWRetFromObs@DerivedMetaData - minvalue: 0. - maxvalue: 999. - - variable: - name: CLWRetFromBkg@DerivedMetaData - minvalue: 0. - maxvalue: 999. - where operator: and - assignments: - - name: CLWRetSymmetric@DerivedMetaData - type: float - function: - name: Arithmetic@ObsFunction - options: - variables: - - name: CLWRetFromObs@DerivedMetaData - - name: CLWRetFromBkg@DerivedMetaData - total coefficient: 0.5 - - # Calculate scattering index from observation - - filter: Variable Assignment - assignments: - - name: SIRetFromObs@DerivedMetaData - type: float - function: - name: SCATRetMW@ObsFunction - options: - scatret_ch238: 1 - scatret_ch314: 2 - scatret_ch890: 16 - scatret_types: [ObsValue] - - # Calculate CLW obs/bkg match index - - filter: Variable Assignment - assignments: - - name: CLWMatchIndex@DerivedMetaData - channels: *{{observation_from_jcb}}_simulated_channels - type: float - function: - name: CLWMatchIndexMW@ObsFunction - channels: *{{observation_from_jcb}}_simulated_channels - options: - channels: *{{observation_from_jcb}}_simulated_channels - clwobs_function: - name: CLWRetFromObs@DerivedMetaData - clwbkg_function: - name: CLWRetFromBkg@DerivedMetaData - clwret_clearsky: [0.030, 0.030, 0.030, 0.020, 0.030, 0.080, 0.150, 0.000, - 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.020, 0.030, 0.030, - 0.030, 0.030, 0.050, 0.100] - - # Calculate symmetric observation error - - filter: Variable Assignment - assignments: - - name: InitialObsError@DerivedMetaData - channels: *{{observation_from_jcb}}_simulated_channels - type: float - function: - name: ObsErrorModelRamp@ObsFunction - channels: *{{observation_from_jcb}}_simulated_channels - options: - channels: *{{observation_from_jcb}}_simulated_channels - xvar: - name: CLWRetSymmetric@DerivedMetaData - x0: [0.030, 0.030, 0.030, 0.020, 0.030, 0.080, 0.150, 0.000, 0.000, 0.000, - 0.000, 0.000, 0.000, 0.000, 0.000, 0.020, 0.030, 0.030, 0.030, 0.030, - 0.050, 0.100] - x1: [0.350, 0.380, 0.400, 0.450, 0.500, 1.000, 1.000, 0.000, 0.000, 0.000, - 0.000, 0.000, 0.000, 0.000, 0.000, 0.350, 0.500, 0.500, 0.500, 0.500, - 0.500, 0.500] - err0: [4.500, 4.500, 4.500, 2.500, 0.550, 0.300, 0.300, 0.400, 0.400, 0.400, - 0.450, 0.450, 0.550, 0.800, 4.000, 4.000, 4.000, 3.500, 3.000, 3.000, - 3.000, 3.000] - err1: [20.000, 25.000, 12.000, 7.000, 3.500, 3.000, 0.800, 0.400, 0.400, - 0.400, 0.450, 0.450, 0.550, 0.800, 4.000, 19.000, 30.000, 25.000, 16.500, - 12.000, 9.000, 6.500] - - # Calculate Innovation@DerivedMetaData - - filter: Variable Assignment - assignments: - - name: Innovation@DerivedMetaData - channels: *{{observation_from_jcb}}_simulated_channels - type: float - function: - name: ObsFunction/Arithmetic - channels: *{{observation_from_jcb}}_simulated_channels - options: - variables: - - name: brightnessTemperature@ObsValue - channels: *{{observation_from_jcb}}_simulated_channels - - name: brightnessTemperature@HofX - channels: *{{observation_from_jcb}}_simulated_channels - coefs: [1, -1] - - # Step 0-C: Assign Initial All-Sky Observation Error - - filter: Perform Action - filter variables: - - name: brightnessTemperature - channels: *{{observation_from_jcb}}_simulated_channels - action: - name: assign error - error function: - name: InitialObsError@DerivedMetaData - channels: *{{observation_from_jcb}}_simulated_channels - - # Step 1: Remove Observations from the Edge of the Scan - - filter: Domain Check - filter variables: - - name: brightnessTemperature - channels: 1-22 - where: - - variable: - name: MetaData/sensorScanPosition - is_in: 7-90 - actions: - - name: set - flag: ScanEdgeRemoval - - name: reject - - # Step 2: Data Thinning - - filter: Gaussian Thinning - horizontal_mesh: 145 - use_reduced_horizontal_grid: true - distance_norm: geodesic - # round_horizontal_bin_count_to_nearest: true - # partition_longitude_bins_using_mesh: true - actions: - - name: set - flag: Thinning - - name: reject - - # Step 3A: CLW Retrieval Check (observation_based) - - filter: Bounds Check - filter variables: - - name: brightnessTemperature - channels: 1-7, 16-22 - test variables: - - name: CLWRetFromObs@DerivedMetaData - maxvalue: 999.0 - actions: - - name: set - flag: CLWRetrievalCheck - - name: reject - - # Step 3B: CLW Retrieval Check (background_based) - - filter: Bounds Check - filter variables: - - name: brightnessTemperature - channels: 1-7, 16-22 - test variables: - - name: CLWRetFromBkg@DerivedMetaData - maxvalue: 999.0 - actions: - - name: set - flag: CLWRetrievalCheck - - name: reject - - # Step 4: Window Channel Sanity Check - - filter: Bounds Check - filter variables: - - name: brightnessTemperature - channels: 1-7, 16, 17-22 - test variables: - - name: Innovation@DerivedMetaData - channels: 1, 2, 5-7, 16 - maxvalue: 200.0 - minvalue: -200.0 - flag all filter variables if any test variable is out of bounds: true - actions: - - name: set - flag: WindowChannelExtremeResidual - - name: reject - - # Step 5: Hydrometeor Check (cloud/precipitation affected chanels) - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/HydrometeorCheckATMS - channels: *{{observation_from_jcb}}_simulated_channels - type: float - function: - name: HydrometeorCheckATMS@ObsFunction - channels: *{{observation_from_jcb}}_simulated_channels - options: - channels: *{{observation_from_jcb}}_simulated_channels - obserr_clearsky: [4.500, 4.500, 4.500, 2.500, 0.550, 0.300, 0.300, 0.400, - 0.400, 0.400, 0.450, 0.450, 0.550, 0.800, 4.000, 4.000, 4.000, 3.500, - 3.000, 3.000, 3.000, 3.000] - clwret_function: - name: CLWRetFromObs@DerivedMetaData - obserr_function: - name: InitialObsError@DerivedMetaData - channels: *{{observation_from_jcb}}_simulated_channels - Clouds: *{{observation_from_jcb}}_clouds - - filter: Bounds Check - filter variables: - - name: brightnessTemperature - channels: *{{observation_from_jcb}}_simulated_channels - test variables: - - name: DerivedMetaData/HydrometeorCheckATMS - channels: *{{observation_from_jcb}}_simulated_channels - maxvalue: 0.0 - actions: - - name: set - flag: HydrometeorCheck - ignore: rejected observations - - name: reject - - # Step 6: Observation Error Inflation based on Topography Check - - filter: Variable Assignment - assignments: - - name: ObsErrorFactorTopo@DerivedMetaData - channels: *{{observation_from_jcb}}_simulated_channels - type: float - function: - name: ObsErrorFactorTopoRad@ObsFunction - channels: *{{observation_from_jcb}}_simulated_channels - options: - sensor: *{{observation_from_jcb}}_sensor_id - channels: *{{observation_from_jcb}}_simulated_channels - - - filter: Perform Action - filter variables: - - name: brightnessTemperature - channels: *{{observation_from_jcb}}_simulated_channels - action: - name: inflate error - inflation variable: - name: ObsErrorFactorTopo@DerivedMetaData - channels: *{{observation_from_jcb}}_simulated_channels - - # Step 7: Obs Error Inflation based on TOA Transmittancec Check - - filter: Variable Assignment - assignments: - - name: ObsErrorFactorTransmitTop@DerivedMetaData - channels: *{{observation_from_jcb}}_simulated_channels - type: float - function: - name: ObsErrorFactorTransmitTopRad@ObsFunction - channels: *{{observation_from_jcb}}_simulated_channels - options: - channels: *{{observation_from_jcb}}_simulated_channels - - - filter: Perform Action - filter variables: - - name: brightnessTemperature - channels: *{{observation_from_jcb}}_simulated_channels - action: - name: inflate error - inflation variable: - name: ObsErrorFactorTransmitTop@DerivedMetaData - channels: *{{observation_from_jcb}}_simulated_channels - - # Step 8: Observation Error Inflation based on Surface Jacobian Check - - filter: Variable Assignment - assignments: - - name: ObsErrorFactorSurfJacobian@DerivedMetaData - channels: *{{observation_from_jcb}}_simulated_channels - type: float - function: - name: ObsErrorFactorSurfJacobianRad@ObsFunction - channels: *{{observation_from_jcb}}_simulated_channels - options: - sensor: *{{observation_from_jcb}}_sensor_id - channels: *{{observation_from_jcb}}_simulated_channels - obserr_demisf: [0.010, 0.020, 0.015, 0.020, 0.200] - obserr_dtempf: [0.500, 2.000, 1.000, 2.000, 4.500] - - - filter: Perform Action - filter variables: - - name: brightnessTemperature - channels: *{{observation_from_jcb}}_simulated_channels - action: - name: inflate error - inflation variable: - name: ObsErrorFactorSurfJacobian@DerivedMetaData - channels: *{{observation_from_jcb}}_simulated_channels - - # Step 9: Situation Dependent Check - - filter: Variable Assignment - assignments: - - name: ObsErrorFactorSituDepend@DerivedMetaData - channels: *{{observation_from_jcb}}_simulated_channels - type: float - function: - name: ObsErrorFactorSituDependMW@ObsFunction - channels: *{{observation_from_jcb}}_simulated_channels - options: - sensor: *{{observation_from_jcb}}_sensor_id - channels: *{{observation_from_jcb}}_simulated_channels - clwbkg_function: - name: CLWRetFromBkg@DerivedMetaData - clwobs_function: - name: CLWRetFromObs@DerivedMetaData - scatobs_function: - name: SIRetFromObs@DerivedMetaData - clwmatchidx_function: - name: CLWMatchIndex@DerivedMetaData - channels: *{{observation_from_jcb}}_simulated_channels - obserr_function: - name: InitialObsError@DerivedMetaData - channels: *{{observation_from_jcb}}_simulated_channels - obserr_clearsky: [4.500, 4.500, 4.500, 2.500, 0.550, 0.300, 0.300, 0.400, - 0.400, 0.400, 0.450, 0.450, 0.550, 0.800, 4.000, 4.000, 4.000, 3.500, - 3.000, 3.000, 3.000, 3.000] - - - filter: Perform Action - filter variables: - - name: brightnessTemperature - channels: *{{observation_from_jcb}}_simulated_channels - action: - name: inflate error - inflation variable: - name: ObsErrorFactorSituDepend@DerivedMetaData - channels: *{{observation_from_jcb}}_simulated_channels - - # Step 10: Gross check - # Remove data if abs(Obs-HofX) > absolute threshold - - filter: Variable Assignment - assignments: - - name: ObsErrorFactorLat@DerivedMetaData - type: float - function: - name: ObsErrorFactorLatRad@ObsFunction - options: - latitude_parameters: [25.0, 0.25, 0.04, 3.0] - - - filter: Variable Assignment - assignments: - - name: ObsErrorBound@DerivedMetaData - channels: *{{observation_from_jcb}}_simulated_channels - type: float - function: - name: ObsErrorBoundMW@ObsFunction - channels: *{{observation_from_jcb}}_simulated_channels - options: - sensor: *{{observation_from_jcb}}_sensor_id - channels: *{{observation_from_jcb}}_simulated_channels - obserr_bound_latitude: - name: ObsErrorFactorLat@DerivedMetaData - obserr_bound_transmittop: - name: ObsErrorFactorTransmitTop@DerivedMetaData - channels: *{{observation_from_jcb}}_simulated_channels - options: - channels: *{{observation_from_jcb}}_simulated_channels - obserr_bound_topo: - name: ObsErrorFactorTopo@DerivedMetaData - channels: *{{observation_from_jcb}}_simulated_channels - obserr_function: - name: InitialObsError@DerivedMetaData - channels: *{{observation_from_jcb}}_simulated_channels - threshold: 3 - threshold_precip: 2.5 - obserr_bound_max: [4.5, 4.5, 3.0, 3.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, - 1.0, 1.0, 2.0, 4.5, 4.5, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0] - - - filter: Background Check - filter variables: - - name: brightnessTemperature - channels: *{{observation_from_jcb}}_simulated_channels - function absolute threshold: - - name: ObsErrorBound@DerivedMetaData - channels: *{{observation_from_jcb}}_simulated_channels - actions: - - name: set - flag: GrossCheck - ignore: rejected observations - - name: reject - - # Step 11: Inter-Channel Check - - filter: Bounds Check - filter variables: - - name: brightnessTemperature - channels: *{{observation_from_jcb}}_simulated_channels - test variables: - - name: InterChannelConsistencyCheck@ObsFunction - channels: *{{observation_from_jcb}}_simulated_channels - options: - channels: *{{observation_from_jcb}}_simulated_channels - use passive_bc: true - sensor: *{{observation_from_jcb}}_sensor_id - use_flag: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 4, 1, 1, 1, 1, 1, 1, - 1] - maxvalue: 1.0e-12 - actions: - - name: set - flag: InterChannelConsistency - ignore: rejected observations - - name: reject - - # Step 12: Useflag Check - - filter: Bounds Check - filter variables: - - name: brightnessTemperature - channels: *{{observation_from_jcb}}_simulated_channels - test variables: - - name: ObsFunction/ChannelUseflagCheckRad - channels: *{{observation_from_jcb}}_simulated_channels - options: - channels: *{{observation_from_jcb}}_simulated_channels - use_flag: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 4, 1, 1, 1, 1, 1, 1, - 1] - minvalue: 1.0e-12 - actions: - - name: set - flag: UseflagCheck - ignore: rejected observations - - name: reject - -# Column liquid cloud - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/ColumnLiquidCloud - type: float - function: - name: TotalColumnCloudGuess@ObsFunction - options: - cloud_content: mass_content_of_cloud_liquid_water_in_atmosphere_layer -# Column ice cloud - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/ColumnIceCloud - type: float - function: - name: TotalColumnCloudGuess@ObsFunction - options: - cloud_content: mass_content_of_cloud_ice_in_atmosphere_layer -# Combined ColumnLiquidCloud + ColumnIceCloud - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/ColumnLiquidIceCloud - type: float - function: - name: ObsFunction/Arithmetic - options: - variables: - - name: DerivedMetaData/ColumnLiquidCloud - - name: DerivedMetaData/ColumnIceCloud - coefs: [1, 1] -# replace zero by 1e-7 in DerivedMetaData/ColumnLiquidIceCloud - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/ColumnLiquidIceCloud - type: float - value: 1.0e-7 - where: - - variable: - name: DerivedMetaData/ColumnLiquidIceCloud - maxvalue: 1.0e-8 - -# ratio of liquid /(Liquid+Ice) - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/ColumnRatioLiquidCloud - type: float - function: - name: ObsFunction/ElementMultiply - options: - variables: - - name: DerivedMetaData/ColumnLiquidCloud - - name: DerivedMetaData/ColumnLiquidIceCloud - exponents: [1,-1] -# TotalColumnVaporGuess - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/TotalColumnVaporGuess - type: float - function: - name: ObsFunction/TotalColumnVaporGuess -# potentialTemperature at surface - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/PotentialTemperatureSurface - type: float - function: - name: PotentialTemperatureGuess2D@ObsFunction - options: - approximate pressure: surface_pressure -# potentialTemperature near 700 hPa - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/PotentialTemperatureAt700hPa - type: float - function: - name: PotentialTemperatureGuess2D@ObsFunction - options: - approximate pressure: 70000.0 -# stability = Difference between PotentialTemperatur values - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/PotentialTemperatureDiff - type: float - function: - name: ObsFunction/Arithmetic - options: - variables: - - name: DerivedMetaData/PotentialTemperatureAt700hPa - - name: DerivedMetaData/PotentialTemperatureSurface - coefs: [1, -1] -## cao_check - - filter: BlackList - filter variables: - - name: brightnessTemperature - channels: 1-7, 16-22 - where: - - variable: - name: ObsFunction/TotalColumnVaporGuess - maxvalue: 8.0 - - variable: - name: DerivedMetaData/ColumnRatioLiquidCloud - maxvalue: 0.5 - - variable: - name: DerivedMetaData/ColumnLiquidIceCloud - minvalue: 1.0e-6 - - variable: - name: DerivedMetaData/PotentialTemperatureDiff - maxvalue: 12.0 - action: - name: reject - # GeoVaLs for Driving Observation Operators (testing mode) - # -------------------------------------------------------- - geovals: - filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" - - # Passed benchmark for UFO testing - # -------------------------------- - passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 index 309da3466..d18985cd8 100644 --- a/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 @@ -22,8 +22,8 @@ obs operator: name: CRTM Absorbers: [H2O, O3, CO2] - Clouds: &{{observation_from_jcb}}_clouds [Water, Ice, Rain, Snow, Graupel] -# Cloud_Fraction: 1.0 + Clouds: [Water, Ice] + Cloud_Fraction: 1.0 Cloud_Seeding: true obs options: Sensor_ID: &{{observation_from_jcb}}_sensor_id atms_npp @@ -37,8 +37,6 @@ obs bias: input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" - variables without bc: [brightnessTemperature] - channels: {{ get_satellite_variable(observation_from_jcb, "not_biascorrtd") }} variational bc: predictors: - name: constant @@ -67,25 +65,6 @@ ratio for small dataset: 2.0 output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" - # ------------------------ - obs prior filters: - # Zero Atmospheric clouds in CRTM where water_area_fraction < 0.99 - - filter: Variable Assignment - assignments: - - name: MetaData/zeroCloudInCRTM - type: int - function: - name: ObsFunction/Conditional - options: -# firstmatchingcase: false - defaultvalue: 0 # Will not zero clouds by default - cases: - - where: - - variable: - name: GeoVaLs/water_area_fraction - maxvalue: 0.99 - value: 1 # Will zero clouds by default - # Observation Pre Filters (QC) # ---------------------------- obs pre filters: @@ -350,7 +329,7 @@ obserr_function: name: InitialObsError@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels - Clouds: *{{observation_from_jcb}}_clouds + - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -474,7 +453,7 @@ channels: *{{observation_from_jcb}}_simulated_channels # Step 10: Gross check - # Remove data if abs(Obs-HofX) > absolute threshold + # Remove data if abs(Obs-HofX) > absolute threhold - filter: Variable Assignment assignments: - name: ObsErrorFactorLat@DerivedMetaData @@ -508,8 +487,7 @@ obserr_function: name: InitialObsError@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels - threshold: 3 - threshold_precip: 2.5 + threhold: 3 obserr_bound_max: [4.5, 4.5, 3.0, 3.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 2.0, 4.5, 4.5, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0] @@ -566,116 +544,7 @@ ignore: rejected observations - name: reject -# Column liquid cloud - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/ColumnLiquidCloud - type: float - function: - name: TotalColumnCloudGuess@ObsFunction - options: - cloud_content: mass_content_of_cloud_liquid_water_in_atmosphere_layer -# Column ice cloud - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/ColumnIceCloud - type: float - function: - name: TotalColumnCloudGuess@ObsFunction - options: - cloud_content: mass_content_of_cloud_ice_in_atmosphere_layer -# Combined ColumnLiquidCloud + ColumnIceCloud - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/ColumnLiquidIceCloud - type: float - function: - name: ObsFunction/Arithmetic - options: - variables: - - name: DerivedMetaData/ColumnLiquidCloud - - name: DerivedMetaData/ColumnIceCloud - coefs: [1, 1] -# replace zero by 1e-7 in DerivedMetaData/ColumnLiquidIceCloud - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/ColumnLiquidIceCloud - type: float - value: 1.0e-7 - where: - - variable: - name: DerivedMetaData/ColumnLiquidIceCloud - maxvalue: 1.0e-8 -# ratio of liquid /(Liquid+Ice) - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/ColumnRatioLiquidCloud - type: float - function: - name: ObsFunction/ElementMultiply - options: - variables: - - name: DerivedMetaData/ColumnLiquidCloud - - name: DerivedMetaData/ColumnLiquidIceCloud - exponents: [1,-1] -# TotalColumnVaporGuess - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/TotalColumnVaporGuess - type: float - function: - name: ObsFunction/TotalColumnVaporGuess -# potentialTemperature at surface - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/PotentialTemperatureSurface - type: float - function: - name: PotentialTemperatureGuess2D@ObsFunction - options: - approximate pressure: surface_pressure -# potentialTemperature near 700 hPa - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/PotentialTemperatureAt700hPa - type: float - function: - name: PotentialTemperatureGuess2D@ObsFunction - options: - approximate pressure: 70000.0 -# stability = Difference between PotentialTemperatur values - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/PotentialTemperatureDiff - type: float - function: - name: ObsFunction/Arithmetic - options: - variables: - - name: DerivedMetaData/PotentialTemperatureAt700hPa - - name: DerivedMetaData/PotentialTemperatureSurface - coefs: [1, -1] -## cao_check - - filter: BlackList - filter variables: - - name: brightnessTemperature - channels: 1-7, 16-22 - where: - - variable: - name: ObsFunction/TotalColumnVaporGuess - maxvalue: 8.0 - - variable: - name: DerivedMetaData/ColumnRatioLiquidCloud - maxvalue: 0.5 - - variable: - name: DerivedMetaData/ColumnLiquidIceCloud - minvalue: 1.0e-6 - - variable: - name: DerivedMetaData/PotentialTemperatureDiff - maxvalue: 12.0 - action: - name: reject # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: From a305935ad45926924f26aefd1f5802fc3280c26b Mon Sep 17 00:00:00 2001 From: Anna Shlyaeva Date: Wed, 22 Jan 2025 11:18:23 -0700 Subject: [PATCH 080/199] Add option for outputting post-processed soca ice increment (#64) Adds yaml section to output postprocessed ("real" aka what get added to CICE restart) sea ice increment. Part of https://github.com/NOAA-EMC/GDASApp/issues/1402 Co-authored-by: Anna Shlyaeva Co-authored-by: Dan Holdaway <27729500+danholdaway@users.noreply.github.com> --- .../algorithm/marine/soca_2cice_global.yaml.j2 | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/parm/jcb-gdas/algorithm/marine/soca_2cice_global.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_2cice_global.yaml.j2 index 01bff96dd..4ccf033cf 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_2cice_global.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_2cice_global.yaml.j2 @@ -29,10 +29,20 @@ variable change: sno_lev: 1 cice output: restart: {{ ice_rst }} + increment output: + datadir: Data + exp: soca2cice + type: incr + date: '{{ fcst_begin }}' + soca increment: + read_from_file: 1 + basename: ./ + ocn_filename: {{ ocn_inc }} + ice_filename: {{ ice_inc }} + date: '{{ fcst_begin }}' output variables: - sea_water_potential_temperature - sea_water_salinity - - sea_water_cell_thickness - sea_ice_area_fraction - sea_ice_thickness - sea_ice_snow_thickness @@ -47,7 +57,6 @@ states: state variables: - sea_water_potential_temperature - sea_water_salinity - - sea_water_cell_thickness - sea_ice_area_fraction - sea_ice_thickness - sea_ice_snow_thickness From 90dd9fb3fb916bcb00bc1167516d8afbe0ff8282 Mon Sep 17 00:00:00 2001 From: Xuanli Li <101414760+XuanliLi-NOAA@users.noreply.github.com> Date: Thu, 23 Jan 2025 16:27:34 -0500 Subject: [PATCH 081/199] Add GNSSRO yaml files (#67) Adding gnssro_**mission**.yaml.j2 data. The GNSS RO data is reorganized so each satellite mission has its own yaml file. --------- Co-authored-by: Dan Holdaway <27729500+danholdaway@users.noreply.github.com> --- .../atmosphere/gnssro_cosmic2.yaml.j2 | 94 +++++++++++++++ .../atmosphere/gnssro_geoopt.yaml.j2 | 96 ++++++++++++++++ .../atmosphere/gnssro_grace.yaml.j2 | 94 +++++++++++++++ .../observations/atmosphere/gnssro_k5.yaml.j2 | 94 +++++++++++++++ .../atmosphere/gnssro_metop.yaml.j2 | 107 ++++++++++++++++++ .../atmosphere/gnssro_paz.yaml.j2 | 94 +++++++++++++++ .../atmosphere/gnssro_piq.yaml.j2 | 96 ++++++++++++++++ .../observations/atmosphere/gnssro_s6.yaml.j2 | 94 +++++++++++++++ .../atmosphere/gnssro_spire.yaml.j2 | 96 ++++++++++++++++ .../atmosphere/gnssro_tdm.yaml.j2 | 94 +++++++++++++++ .../atmosphere/gnssro_tsx.yaml.j2 | 94 +++++++++++++++ 11 files changed, 1053 insertions(+) create mode 100644 parm/jcb-gdas/observations/atmosphere/gnssro_cosmic2.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/gnssro_geoopt.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/gnssro_grace.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/gnssro_k5.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/gnssro_metop.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/gnssro_paz.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/gnssro_piq.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/gnssro_s6.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/gnssro_spire.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/gnssro_tdm.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/gnssro_tsx.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_cosmic2.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_cosmic2.yaml.j2 new file mode 100644 index 000000000..8995da282 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_cosmic2.yaml.j2 @@ -0,0 +1,94 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: gnssrobndnbam + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsgrouping: + group variables: [ 'sequenceNumber' ] + sort variable: 'impactHeightRO' + sort order: 'ascending' + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + simulated variables: [bendingAngle] + + # Observation Operator + # -------------------- + obs operator: + name: GnssroBndNBAM + obs options: + use_compress: 1 + sr_steps: 2 + vertlayer: full + super_ref_qc: NBAM + output_diags: true + + # Observation Filters (QC) + # ------------------------ + obs filters: + # Apply gross check using pccf + # Step 0-A: Create Diagnostic Flags + # Diagnostic flag for pccf + - filter: Create Diagnostic Flags + filter variables: + - name: bendingAngle + flags: + - name: pccfCheckReject + initial value: false + force reinitialization: true + + # Step 0-B: pccf Check for CDACC data - good: 0.1-100, reject: 0 + - filter: Bounds Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/satelliteIdentifier + is_in: 265-269,750-755,44,786,820,825 + test variables: + - name: MetaData/pccf + minvalue: 0.1 + maxvalue: 100.1 + actions: + - name: set + flag: pccfCheckReject + - name: reject + + #1. gpstop + - filter: Domain Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/impactHeightRO + minvalue: 0 + maxvalue: 55000.1 + action: + name: reject + #3. RONBAM cut off check + - filter: Background Check RONBAM + filter variables: + - name: bendingAngle + action: + name: reject + defer to post: true + #4. assign obs error + - filter: ROobserror + filter variables: + - name: bendingAngle + errmodel: NBAM + defer to post: true + #5. Obs error inflate + - filter: Perform Action + filter variables: + - name: bendingAngle + action: + name: RONBAMErrInflate + defer to post: true + # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_geoopt.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_geoopt.yaml.j2 new file mode 100644 index 000000000..67504fa54 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_geoopt.yaml.j2 @@ -0,0 +1,96 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: gnssrobndnbam + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsgrouping: + group variables: [ 'sequenceNumber' ] + sort variable: 'impactHeightRO' + sort order: 'ascending' + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + simulated variables: [bendingAngle] + + # Observation Operator + # -------------------- + obs operator: + name: GnssroBndNBAM + obs options: + use_compress: 1 + sr_steps: 2 + vertlayer: full + super_ref_qc: NBAM + output_diags: true + + # Observation Filters (QC) + # ------------------------ + obs filters: + # Apply gross check using pccf + # Step 0-A: Create Diagnostic Flags + # Diagnostic flag for pccf + - filter: Create Diagnostic Flags + filter variables: + - name: bendingAngle + flags: + - name: pccfCheckReject + initial value: false + force reinitialization: true + + # Step 0-B: pccf Check for CDACC data - good: 0.1-100, reject: 0 + - filter: Bounds Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/satelliteIdentifier + is_in: 265-269,750-755,44,786,820,825 + test variables: + - name: MetaData/pccf + minvalue: 0.1 + maxvalue: 100.1 + actions: + - name: set + flag: pccfCheckReject + - name: reject + + #1. commgpstop + - filter: Bounds Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/satelliteIdentifier + is_in: 265,266,267,268,269 + test variables: + - name: MetaData/impactHeightRO + maxvalue: 45000.1 + action: + name: reject + #3. RONBAM cut off check + - filter: Background Check RONBAM + filter variables: + - name: bendingAngle + action: + name: reject + defer to post: true + #4. assign obs error + - filter: ROobserror + filter variables: + - name: bendingAngle + errmodel: NBAM + defer to post: true + #5. Obs error inflate + - filter: Perform Action + filter variables: + - name: bendingAngle + action: + name: RONBAMErrInflate + defer to post: true + # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_grace.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_grace.yaml.j2 new file mode 100644 index 000000000..8995da282 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_grace.yaml.j2 @@ -0,0 +1,94 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: gnssrobndnbam + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsgrouping: + group variables: [ 'sequenceNumber' ] + sort variable: 'impactHeightRO' + sort order: 'ascending' + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + simulated variables: [bendingAngle] + + # Observation Operator + # -------------------- + obs operator: + name: GnssroBndNBAM + obs options: + use_compress: 1 + sr_steps: 2 + vertlayer: full + super_ref_qc: NBAM + output_diags: true + + # Observation Filters (QC) + # ------------------------ + obs filters: + # Apply gross check using pccf + # Step 0-A: Create Diagnostic Flags + # Diagnostic flag for pccf + - filter: Create Diagnostic Flags + filter variables: + - name: bendingAngle + flags: + - name: pccfCheckReject + initial value: false + force reinitialization: true + + # Step 0-B: pccf Check for CDACC data - good: 0.1-100, reject: 0 + - filter: Bounds Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/satelliteIdentifier + is_in: 265-269,750-755,44,786,820,825 + test variables: + - name: MetaData/pccf + minvalue: 0.1 + maxvalue: 100.1 + actions: + - name: set + flag: pccfCheckReject + - name: reject + + #1. gpstop + - filter: Domain Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/impactHeightRO + minvalue: 0 + maxvalue: 55000.1 + action: + name: reject + #3. RONBAM cut off check + - filter: Background Check RONBAM + filter variables: + - name: bendingAngle + action: + name: reject + defer to post: true + #4. assign obs error + - filter: ROobserror + filter variables: + - name: bendingAngle + errmodel: NBAM + defer to post: true + #5. Obs error inflate + - filter: Perform Action + filter variables: + - name: bendingAngle + action: + name: RONBAMErrInflate + defer to post: true + # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_k5.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_k5.yaml.j2 new file mode 100644 index 000000000..8995da282 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_k5.yaml.j2 @@ -0,0 +1,94 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: gnssrobndnbam + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsgrouping: + group variables: [ 'sequenceNumber' ] + sort variable: 'impactHeightRO' + sort order: 'ascending' + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + simulated variables: [bendingAngle] + + # Observation Operator + # -------------------- + obs operator: + name: GnssroBndNBAM + obs options: + use_compress: 1 + sr_steps: 2 + vertlayer: full + super_ref_qc: NBAM + output_diags: true + + # Observation Filters (QC) + # ------------------------ + obs filters: + # Apply gross check using pccf + # Step 0-A: Create Diagnostic Flags + # Diagnostic flag for pccf + - filter: Create Diagnostic Flags + filter variables: + - name: bendingAngle + flags: + - name: pccfCheckReject + initial value: false + force reinitialization: true + + # Step 0-B: pccf Check for CDACC data - good: 0.1-100, reject: 0 + - filter: Bounds Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/satelliteIdentifier + is_in: 265-269,750-755,44,786,820,825 + test variables: + - name: MetaData/pccf + minvalue: 0.1 + maxvalue: 100.1 + actions: + - name: set + flag: pccfCheckReject + - name: reject + + #1. gpstop + - filter: Domain Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/impactHeightRO + minvalue: 0 + maxvalue: 55000.1 + action: + name: reject + #3. RONBAM cut off check + - filter: Background Check RONBAM + filter variables: + - name: bendingAngle + action: + name: reject + defer to post: true + #4. assign obs error + - filter: ROobserror + filter variables: + - name: bendingAngle + errmodel: NBAM + defer to post: true + #5. Obs error inflate + - filter: Perform Action + filter variables: + - name: bendingAngle + action: + name: RONBAMErrInflate + defer to post: true + # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_metop.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_metop.yaml.j2 new file mode 100644 index 000000000..7842fdc2b --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_metop.yaml.j2 @@ -0,0 +1,107 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: gnssrobndnbam + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsgrouping: + group variables: [ 'sequenceNumber' ] + sort variable: 'impactHeightRO' + sort order: 'ascending' + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + simulated variables: [bendingAngle] + + # Observation Operator + # -------------------- + obs operator: + name: GnssroBndNBAM + obs options: + use_compress: 1 + sr_steps: 2 + vertlayer: full + super_ref_qc: NBAM + output_diags: true + + # Observation Filters (QC) + # ------------------------ + obs filters: + # Apply gross check using qfro + # Step 0-A: Create Diagnostic Flags + # Diagnostic flag for qfro + - filter: Create Diagnostic Flags + filter variables: + - name: bendingAngle + flags: + - name: qfroCheckReject + initial value: false + force reinitialization: true + + # Step 0-B: qfro Check - good: 0, reject: 1 + - filter: Bounds Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/satelliteIdentifier + is_in: 3-5,421,440,821 + test variables: + - name: MetaData/qualityFlags + minvalue: -0.1 + maxvalue: 0.9 + actions: + - name: set + flag: qfroCheckReject + - name: reject + + #1. gpstop + - filter: Domain Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/impactHeightRO + minvalue: 0 + maxvalue: 55000.1 + action: + name: reject + #2. metop below 8 km + - filter: Bounds Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/satelliteIdentifier + is_in: 3-5 + test variables: + - name: MetaData/impactHeightRO + minvalue: 8000.1 + action: + name: reject + #3. RONBAM cut off check + - filter: Background Check RONBAM + filter variables: + - name: bendingAngle + action: + name: reject + defer to post: true + #4. assign obs error + - filter: ROobserror + filter variables: + - name: bendingAngle + errmodel: NBAM + defer to post: true + #5. Obs error inflate + - filter: Perform Action + filter variables: + - name: bendingAngle + action: + name: RONBAMErrInflate + defer to post: true + # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_paz.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_paz.yaml.j2 new file mode 100644 index 000000000..8995da282 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_paz.yaml.j2 @@ -0,0 +1,94 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: gnssrobndnbam + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsgrouping: + group variables: [ 'sequenceNumber' ] + sort variable: 'impactHeightRO' + sort order: 'ascending' + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + simulated variables: [bendingAngle] + + # Observation Operator + # -------------------- + obs operator: + name: GnssroBndNBAM + obs options: + use_compress: 1 + sr_steps: 2 + vertlayer: full + super_ref_qc: NBAM + output_diags: true + + # Observation Filters (QC) + # ------------------------ + obs filters: + # Apply gross check using pccf + # Step 0-A: Create Diagnostic Flags + # Diagnostic flag for pccf + - filter: Create Diagnostic Flags + filter variables: + - name: bendingAngle + flags: + - name: pccfCheckReject + initial value: false + force reinitialization: true + + # Step 0-B: pccf Check for CDACC data - good: 0.1-100, reject: 0 + - filter: Bounds Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/satelliteIdentifier + is_in: 265-269,750-755,44,786,820,825 + test variables: + - name: MetaData/pccf + minvalue: 0.1 + maxvalue: 100.1 + actions: + - name: set + flag: pccfCheckReject + - name: reject + + #1. gpstop + - filter: Domain Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/impactHeightRO + minvalue: 0 + maxvalue: 55000.1 + action: + name: reject + #3. RONBAM cut off check + - filter: Background Check RONBAM + filter variables: + - name: bendingAngle + action: + name: reject + defer to post: true + #4. assign obs error + - filter: ROobserror + filter variables: + - name: bendingAngle + errmodel: NBAM + defer to post: true + #5. Obs error inflate + - filter: Perform Action + filter variables: + - name: bendingAngle + action: + name: RONBAMErrInflate + defer to post: true + # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_piq.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_piq.yaml.j2 new file mode 100644 index 000000000..67504fa54 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_piq.yaml.j2 @@ -0,0 +1,96 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: gnssrobndnbam + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsgrouping: + group variables: [ 'sequenceNumber' ] + sort variable: 'impactHeightRO' + sort order: 'ascending' + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + simulated variables: [bendingAngle] + + # Observation Operator + # -------------------- + obs operator: + name: GnssroBndNBAM + obs options: + use_compress: 1 + sr_steps: 2 + vertlayer: full + super_ref_qc: NBAM + output_diags: true + + # Observation Filters (QC) + # ------------------------ + obs filters: + # Apply gross check using pccf + # Step 0-A: Create Diagnostic Flags + # Diagnostic flag for pccf + - filter: Create Diagnostic Flags + filter variables: + - name: bendingAngle + flags: + - name: pccfCheckReject + initial value: false + force reinitialization: true + + # Step 0-B: pccf Check for CDACC data - good: 0.1-100, reject: 0 + - filter: Bounds Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/satelliteIdentifier + is_in: 265-269,750-755,44,786,820,825 + test variables: + - name: MetaData/pccf + minvalue: 0.1 + maxvalue: 100.1 + actions: + - name: set + flag: pccfCheckReject + - name: reject + + #1. commgpstop + - filter: Bounds Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/satelliteIdentifier + is_in: 265,266,267,268,269 + test variables: + - name: MetaData/impactHeightRO + maxvalue: 45000.1 + action: + name: reject + #3. RONBAM cut off check + - filter: Background Check RONBAM + filter variables: + - name: bendingAngle + action: + name: reject + defer to post: true + #4. assign obs error + - filter: ROobserror + filter variables: + - name: bendingAngle + errmodel: NBAM + defer to post: true + #5. Obs error inflate + - filter: Perform Action + filter variables: + - name: bendingAngle + action: + name: RONBAMErrInflate + defer to post: true + # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_s6.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_s6.yaml.j2 new file mode 100644 index 000000000..8995da282 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_s6.yaml.j2 @@ -0,0 +1,94 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: gnssrobndnbam + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsgrouping: + group variables: [ 'sequenceNumber' ] + sort variable: 'impactHeightRO' + sort order: 'ascending' + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + simulated variables: [bendingAngle] + + # Observation Operator + # -------------------- + obs operator: + name: GnssroBndNBAM + obs options: + use_compress: 1 + sr_steps: 2 + vertlayer: full + super_ref_qc: NBAM + output_diags: true + + # Observation Filters (QC) + # ------------------------ + obs filters: + # Apply gross check using pccf + # Step 0-A: Create Diagnostic Flags + # Diagnostic flag for pccf + - filter: Create Diagnostic Flags + filter variables: + - name: bendingAngle + flags: + - name: pccfCheckReject + initial value: false + force reinitialization: true + + # Step 0-B: pccf Check for CDACC data - good: 0.1-100, reject: 0 + - filter: Bounds Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/satelliteIdentifier + is_in: 265-269,750-755,44,786,820,825 + test variables: + - name: MetaData/pccf + minvalue: 0.1 + maxvalue: 100.1 + actions: + - name: set + flag: pccfCheckReject + - name: reject + + #1. gpstop + - filter: Domain Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/impactHeightRO + minvalue: 0 + maxvalue: 55000.1 + action: + name: reject + #3. RONBAM cut off check + - filter: Background Check RONBAM + filter variables: + - name: bendingAngle + action: + name: reject + defer to post: true + #4. assign obs error + - filter: ROobserror + filter variables: + - name: bendingAngle + errmodel: NBAM + defer to post: true + #5. Obs error inflate + - filter: Perform Action + filter variables: + - name: bendingAngle + action: + name: RONBAMErrInflate + defer to post: true + # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_spire.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_spire.yaml.j2 new file mode 100644 index 000000000..67504fa54 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_spire.yaml.j2 @@ -0,0 +1,96 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: gnssrobndnbam + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsgrouping: + group variables: [ 'sequenceNumber' ] + sort variable: 'impactHeightRO' + sort order: 'ascending' + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + simulated variables: [bendingAngle] + + # Observation Operator + # -------------------- + obs operator: + name: GnssroBndNBAM + obs options: + use_compress: 1 + sr_steps: 2 + vertlayer: full + super_ref_qc: NBAM + output_diags: true + + # Observation Filters (QC) + # ------------------------ + obs filters: + # Apply gross check using pccf + # Step 0-A: Create Diagnostic Flags + # Diagnostic flag for pccf + - filter: Create Diagnostic Flags + filter variables: + - name: bendingAngle + flags: + - name: pccfCheckReject + initial value: false + force reinitialization: true + + # Step 0-B: pccf Check for CDACC data - good: 0.1-100, reject: 0 + - filter: Bounds Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/satelliteIdentifier + is_in: 265-269,750-755,44,786,820,825 + test variables: + - name: MetaData/pccf + minvalue: 0.1 + maxvalue: 100.1 + actions: + - name: set + flag: pccfCheckReject + - name: reject + + #1. commgpstop + - filter: Bounds Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/satelliteIdentifier + is_in: 265,266,267,268,269 + test variables: + - name: MetaData/impactHeightRO + maxvalue: 45000.1 + action: + name: reject + #3. RONBAM cut off check + - filter: Background Check RONBAM + filter variables: + - name: bendingAngle + action: + name: reject + defer to post: true + #4. assign obs error + - filter: ROobserror + filter variables: + - name: bendingAngle + errmodel: NBAM + defer to post: true + #5. Obs error inflate + - filter: Perform Action + filter variables: + - name: bendingAngle + action: + name: RONBAMErrInflate + defer to post: true + # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_tdm.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_tdm.yaml.j2 new file mode 100644 index 000000000..8995da282 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_tdm.yaml.j2 @@ -0,0 +1,94 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: gnssrobndnbam + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsgrouping: + group variables: [ 'sequenceNumber' ] + sort variable: 'impactHeightRO' + sort order: 'ascending' + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + simulated variables: [bendingAngle] + + # Observation Operator + # -------------------- + obs operator: + name: GnssroBndNBAM + obs options: + use_compress: 1 + sr_steps: 2 + vertlayer: full + super_ref_qc: NBAM + output_diags: true + + # Observation Filters (QC) + # ------------------------ + obs filters: + # Apply gross check using pccf + # Step 0-A: Create Diagnostic Flags + # Diagnostic flag for pccf + - filter: Create Diagnostic Flags + filter variables: + - name: bendingAngle + flags: + - name: pccfCheckReject + initial value: false + force reinitialization: true + + # Step 0-B: pccf Check for CDACC data - good: 0.1-100, reject: 0 + - filter: Bounds Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/satelliteIdentifier + is_in: 265-269,750-755,44,786,820,825 + test variables: + - name: MetaData/pccf + minvalue: 0.1 + maxvalue: 100.1 + actions: + - name: set + flag: pccfCheckReject + - name: reject + + #1. gpstop + - filter: Domain Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/impactHeightRO + minvalue: 0 + maxvalue: 55000.1 + action: + name: reject + #3. RONBAM cut off check + - filter: Background Check RONBAM + filter variables: + - name: bendingAngle + action: + name: reject + defer to post: true + #4. assign obs error + - filter: ROobserror + filter variables: + - name: bendingAngle + errmodel: NBAM + defer to post: true + #5. Obs error inflate + - filter: Perform Action + filter variables: + - name: bendingAngle + action: + name: RONBAMErrInflate + defer to post: true + # -------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_tsx.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_tsx.yaml.j2 new file mode 100644 index 000000000..8995da282 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_tsx.yaml.j2 @@ -0,0 +1,94 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: gnssrobndnbam + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsgrouping: + group variables: [ 'sequenceNumber' ] + sort variable: 'impactHeightRO' + sort order: 'ascending' + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + simulated variables: [bendingAngle] + + # Observation Operator + # -------------------- + obs operator: + name: GnssroBndNBAM + obs options: + use_compress: 1 + sr_steps: 2 + vertlayer: full + super_ref_qc: NBAM + output_diags: true + + # Observation Filters (QC) + # ------------------------ + obs filters: + # Apply gross check using pccf + # Step 0-A: Create Diagnostic Flags + # Diagnostic flag for pccf + - filter: Create Diagnostic Flags + filter variables: + - name: bendingAngle + flags: + - name: pccfCheckReject + initial value: false + force reinitialization: true + + # Step 0-B: pccf Check for CDACC data - good: 0.1-100, reject: 0 + - filter: Bounds Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/satelliteIdentifier + is_in: 265-269,750-755,44,786,820,825 + test variables: + - name: MetaData/pccf + minvalue: 0.1 + maxvalue: 100.1 + actions: + - name: set + flag: pccfCheckReject + - name: reject + + #1. gpstop + - filter: Domain Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/impactHeightRO + minvalue: 0 + maxvalue: 55000.1 + action: + name: reject + #3. RONBAM cut off check + - filter: Background Check RONBAM + filter variables: + - name: bendingAngle + action: + name: reject + defer to post: true + #4. assign obs error + - filter: ROobserror + filter variables: + - name: bendingAngle + errmodel: NBAM + defer to post: true + #5. Obs error inflate + - filter: Perform Action + filter variables: + - name: bendingAngle + action: + name: RONBAMErrInflate + defer to post: true + # -------------------------------- From 36838bfa1319a105fd27005996d0099447a7b9ca Mon Sep 17 00:00:00 2001 From: Xuanli Li <101414760+XuanliLi-NOAA@users.noreply.github.com> Date: Fri, 24 Jan 2025 11:38:00 -0500 Subject: [PATCH 082/199] Feature/gnssroyaml (#69) Update the obs space name for the gnssro yaml files. --- parm/jcb-gdas/observations/atmosphere/gnssro_cosmic2.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/gnssro_geoopt.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/gnssro_grace.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/gnssro_k5.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/gnssro_metop.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/gnssro_paz.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/gnssro_piq.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/gnssro_s6.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/gnssro_spire.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/gnssro_tdm.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/gnssro_tsx.yaml.j2 | 2 +- 11 files changed, 11 insertions(+), 11 deletions(-) diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_cosmic2.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_cosmic2.yaml.j2 index 8995da282..fa3f48869 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_cosmic2.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_cosmic2.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: gnssrobndnbam + name: gnssrobndnbam_cosmic2 obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_geoopt.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_geoopt.yaml.j2 index 67504fa54..a7774c545 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_geoopt.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_geoopt.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: gnssrobndnbam + name: gnssrobndnbam_geoopt obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_grace.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_grace.yaml.j2 index 8995da282..5a459de8d 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_grace.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_grace.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: gnssrobndnbam + name: gnssrobndnbam_grace obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_k5.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_k5.yaml.j2 index 8995da282..72893f481 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_k5.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_k5.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: gnssrobndnbam + name: gnssrobndnbam_k5 obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_metop.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_metop.yaml.j2 index 7842fdc2b..199424b53 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_metop.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_metop.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: gnssrobndnbam + name: gnssrobndnbam_metop obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_paz.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_paz.yaml.j2 index 8995da282..c36e8923a 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_paz.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_paz.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: gnssrobndnbam + name: gnssrobndnbam_paz obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_piq.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_piq.yaml.j2 index 67504fa54..ad4686b9f 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_piq.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_piq.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: gnssrobndnbam + name: gnssrobndnbam_piq obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_s6.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_s6.yaml.j2 index 8995da282..bbb0698b1 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_s6.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_s6.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: gnssrobndnbam + name: gnssrobndnbam_s6 obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_spire.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_spire.yaml.j2 index 67504fa54..568da6de7 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_spire.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_spire.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: gnssrobndnbam + name: gnssrobndnbam_spire obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_tdm.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_tdm.yaml.j2 index 8995da282..fb38f0126 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_tdm.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_tdm.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: gnssrobndnbam + name: gnssrobndnbam_tdm obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_tsx.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_tsx.yaml.j2 index 8995da282..107308486 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_tsx.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_tsx.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: gnssrobndnbam + name: gnssrobndnbam_tsx obsdatain: engine: type: H5File From 4ce5ea7dd919735c75b7ef61ea008244e3d4e7e0 Mon Sep 17 00:00:00 2001 From: jianjunj Date: Fri, 24 Jan 2025 13:39:27 -0500 Subject: [PATCH 083/199] Set channels without bias correction. (#59) Set channels without bias correction from chronicle tables. ATMS Channel 15 and AMSUA channel 14 are bias corrected. Set -999 in the channels list "without bc" to let UFO conduct bias correction for all channels. Co-authored-by: Dan Holdaway <27729500+danholdaway@users.noreply.github.com> --- parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 | 2 ++ 1 file changed, 2 insertions(+) diff --git a/parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 index 6bea2e3cc..e0737160e 100644 --- a/parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 @@ -34,6 +34,8 @@ obs bias: input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" + variables without bc: [brightnessTemperature] + channels: &{{observation_from_jcb}}_not_bias_corrected {{ get_satellite_variable(observation_from_jcb, "not_biascorrtd") }} variational bc: predictors: - name: constant From 36c0130e5c520089044e2ab08ce2864aac5ceee4 Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Tue, 28 Jan 2025 17:19:24 +0000 Subject: [PATCH 084/199] change aerosol gaussian increment from ges res to anl res (#71) OOPS chokes trying to interpolate and write a C1152 equivalent Gaussian increment, and we don't need that anyways for now, so instead write out an increment at the analysis resolution (C384) --- parm/jcb-gdas/model/aero/aero_final_increment_gaussian.yaml.j2 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/parm/jcb-gdas/model/aero/aero_final_increment_gaussian.yaml.j2 b/parm/jcb-gdas/model/aero/aero_final_increment_gaussian.yaml.j2 index 97ffc8d4a..72eabe067 100644 --- a/parm/jcb-gdas/model/aero/aero_final_increment_gaussian.yaml.j2 +++ b/parm/jcb-gdas/model/aero/aero_final_increment_gaussian.yaml.j2 @@ -2,7 +2,7 @@ grid type: regular gaussian local interpolator type: atlas interpolator interpolation method: type: finite-element -number of latitude gridpoints: {{ aero_npy_ges - 1 }} +number of latitude gridpoints: {{ aero_npy_anl - 1 }} variables to output: [mass_fraction_of_sulfate_in_air, mass_fraction_of_hydrophobic_black_carbon_in_air, mass_fraction_of_hydrophilic_black_carbon_in_air, From c0dae219a52deb81ca2ddb4d6d682a768eddfd09 Mon Sep 17 00:00:00 2001 From: Guillaume Vernieres Date: Mon, 3 Feb 2025 11:15:49 -0500 Subject: [PATCH 085/199] Configuration for the low res marine B (#66) description coming soon ... Resolves part of https://github.com/NOAA-EMC/GDASApp/issues/1375 --------- Co-authored-by: Dan Holdaway <27729500+danholdaway@users.noreply.github.com> --- .../algorithm/marine/soca_chgres.yaml.j2 | 33 +++++++++ .../algorithm/marine/soca_diagb.yaml.j2 | 5 ++ .../algorithm/marine/soca_ensb.yaml.j2 | 7 +- .../algorithm/marine/soca_ensweights.yaml.j2 | 24 +++--- .../soca_parameters_diffusion_hz.yaml.j2 | 11 ++- .../soca_parameters_diffusion_vt.yaml.j2 | 12 +-- .../marine/soca_setcorscales.yaml.j2 | 7 +- .../algorithm/marine/soca_vtscales.yaml.j2 | 6 +- .../algorithm/marine/socaincr2mom6.yaml.j2 | 2 + .../marine/marine_3dfgat_outer_loop_1.yaml.j2 | 3 +- .../marine/marine_3dvar_outer_loop_1.yaml.j2 | 7 -- ...d_error_hybrid_diffusion_diffusion.yaml.j2 | 74 ++++++++++--------- ..._background_error_static_diffusion.yaml.j2 | 1 + .../gdas-marine-templates.yaml | 1 - 14 files changed, 124 insertions(+), 69 deletions(-) create mode 100644 parm/jcb-gdas/algorithm/marine/soca_chgres.yaml.j2 delete mode 100644 parm/jcb-gdas/model/marine/marine_3dvar_outer_loop_1.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/marine/soca_chgres.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_chgres.yaml.j2 new file mode 100644 index 000000000..2f417bc29 --- /dev/null +++ b/parm/jcb-gdas/algorithm/marine/soca_chgres.yaml.j2 @@ -0,0 +1,33 @@ +input geometry: + mom6_input_nml: mom_input.nml + fields metadata: ./fields_metadata.yaml + +output geometry: + geom_grid_file: ./anl_geom/soca_gridspec.nc + mom6_input_nml: ./anl_geom/mom_input.nml + fields metadata: ./fields_metadata.yaml + +states: +- input: + date: '{{ marine_window_end_iso }}' + basename: ./bkg/ + ocn_filename: 'ocean.bkg.f009.nc' + ice_filename: 'ice.bkg.f009.nc' + read_from_file: 1 + + state variables: + - sea_surface_height_above_geoid + - sea_water_potential_temperature + - sea_water_salinity + - sea_water_cell_thickness + - sea_water_depth + - mom6_mld + - sea_ice_area_fraction + - sea_ice_thickness + - sea_ice_snow_thickness + + output: + datadir: anl_geom/ + exp: f009_anl_geom + type: fc + date: '{{ marine_window_end_iso }}' \ No newline at end of file diff --git a/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 index 788f674de..1f7c0ebe5 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 @@ -2,6 +2,11 @@ geometry: mom6_input_nml: mom_input.nml fields metadata: ./fields_metadata.yaml +output geometry: + geom_grid_file: ./anl_geom/soca_gridspec.nc + mom6_input_nml: ./anl_geom/mom_input.nml + fields metadata: ./fields_metadata.yaml + date: '{{ marine_window_end_iso }}' background: diff --git a/parm/jcb-gdas/algorithm/marine/soca_ensb.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_ensb.yaml.j2 index 852850f15..ab437fafa 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_ensb.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_ensb.yaml.j2 @@ -3,6 +3,11 @@ geometry: mom6_input_nml: mom_input.nml fields metadata: ./fields_metadata.yaml +output geometry: + geom_grid_file: ./anl_geom/soca_gridspec.nc + mom6_input_nml: ./anl_geom/mom_input.nml + fields metadata: ./fields_metadata.yaml + date: '{{ marine_window_begin_iso }}' layers variable: [sea_water_cell_thickness] @@ -113,5 +118,5 @@ output increment: date: '{{ marine_window_begin_iso }}' exp: trash type: incr - output file: 'ocn.pert.steric.%mem%.nc' + output file: 'pert.%mem%.nc' pattern: '%mem%' diff --git a/parm/jcb-gdas/algorithm/marine/soca_ensweights.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_ensweights.yaml.j2 index 9f818b34f..00b2e1ef8 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_ensweights.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_ensweights.yaml.j2 @@ -2,6 +2,11 @@ geometry: mom6_input_nml: mom_input.nml fields metadata: ./fields_metadata.yaml +output geometry: + geom_grid_file: ./anl_geom/soca_gridspec.nc + mom6_input_nml: ./anl_geom/mom_input.nml + fields metadata: ./fields_metadata.yaml + date: '{{ marine_window_middle_iso }}' variables: @@ -29,15 +34,16 @@ weights: ice: 0.0025 # 5% of original variance ocean: 0.0625 # 25% " " # Apply localized weights to the ocean ens. B - ocean local weights: - - lon: -172.0 - lat: 11.0 - amplitude: -1.0 - length scale: 700.0 - - lon: -160.0 - lat: 12.0 - amplitude: -1.0 - length scale: 700.0 + # Example below was for old ens. from Xiao + # ocean local weights: + # - lon: -172.0 + # lat: 11.0 + # amplitude: -1.0 + # length scale: 700.0 + # - lon: -160.0 + # lat: 12.0 + # amplitude: -1.0 + # length scale: 700.0 output: datadir: ./ diff --git a/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_hz.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_hz.yaml.j2 index 8f3cdf798..cb57948ab 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_hz.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_hz.yaml.j2 @@ -1,12 +1,15 @@ +# Analysis/B-mat resolution geometry: &geom - mom6_input_nml: mom_input.nml + geom_grid_file: ./anl_geom/soca_gridspec.nc + mom6_input_nml: ./anl_geom/mom_input.nml fields metadata: ./fields_metadata.yaml +# Background interpolated at the analysis resolution background: read_from_file: 1 - basename: ./INPUT/ - ocn_filename: MOM.res.nc - ice_filename: cice.res.nc + basename: ./anl_geom/ + ocn_filename: 'ocn.f009_anl_geom.fc.{{ marine_window_end_iso }}.PT0S.nc' + ice_filename: 'ice.f009_anl_geom.fc.{{ marine_window_end_iso }}.PT0S.nc' date: '{{ marine_window_end_iso }}' state variables: [sea_surface_height_above_geoid] diff --git a/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_vt.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_vt.yaml.j2 index 8e23f2d9b..a5c5d4641 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_vt.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_parameters_diffusion_vt.yaml.j2 @@ -1,13 +1,15 @@ +# Analysis/B-mat resolution geometry: &geom - mom6_input_nml: mom_input.nml + geom_grid_file: ./anl_geom/soca_gridspec.nc + mom6_input_nml: ./anl_geom/mom_input.nml fields metadata: ./fields_metadata.yaml +# Background interpolated at the analysis resolution background: read_from_file: 1 - basename: ./INPUT/ - ocn_filename: MOM.res.nc - ice_filename: cice.res.nc - date: '{{ marine_window_middle_iso }}' + basename: ./anl_geom/ + ocn_filename: 'ocn.f009_anl_geom.fc.{{ marine_window_end_iso }}.PT0S.nc' + date: '{{ marine_window_end_iso }}' state variables: [sea_water_potential_temperature] background error: diff --git a/parm/jcb-gdas/algorithm/marine/soca_setcorscales.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_setcorscales.yaml.j2 index 0c2b9c06b..9d07bef08 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_setcorscales.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_setcorscales.yaml.j2 @@ -1,5 +1,6 @@ resolution: - mom6_input_nml: mom_input.nml + geom_grid_file: ./anl_geom/soca_gridspec.nc + mom6_input_nml: ./anl_geom/mom_input.nml fields metadata: ./fields_metadata.yaml date: "0001-01-01T00:00:00Z" @@ -9,8 +10,8 @@ corr variables: [sea_surface_height_above_geoid] scales: vert layers: 5 # in units of layer sea_surface_height_above_geoid: - rossby mult: 2.00 - min grid mult: 6.0 + rossby mult: 1.50 + min grid mult: 2.0 rh output: datadir: ./ diff --git a/parm/jcb-gdas/algorithm/marine/soca_vtscales.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_vtscales.yaml.j2 index 797af37c3..f0e68bc15 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_vtscales.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_vtscales.yaml.j2 @@ -1,6 +1,6 @@ -gridspec_filename: soca_gridspec.nc -restart_filename: ./INPUT/MOM.res.nc -mld_filename: './staticb/ocn.bkgerr_stddev.incr.{{ marine_window_end_iso }}.nc' +gridspec_filename: ./anl_geom/soca_gridspec.nc +restart_filename: './anl_geom/ocn.f009_anl_geom.fc.{{ marine_window_end_iso }}.PT0S.nc' +mld_filename: './anl_geom/ocn.f009_anl_geom.fc.{{ marine_window_end_iso }}.PT0S.nc' output_filename: ./vt_scales.nc output_variable_vt: Temp output_variable_hz: ave_ssh diff --git a/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 b/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 index 0790fc9c5..85b8f0396 100644 --- a/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 @@ -6,6 +6,8 @@ date: '{{ marine_window_begin }}' layers variable: [sea_water_cell_thickness] +domains: [ocn] + increment variables: - sea_water_potential_temperature - sea_water_salinity diff --git a/parm/jcb-gdas/model/marine/marine_3dfgat_outer_loop_1.yaml.j2 b/parm/jcb-gdas/model/marine/marine_3dfgat_outer_loop_1.yaml.j2 index 2931f62a1..e90d9ddc0 100644 --- a/parm/jcb-gdas/model/marine/marine_3dfgat_outer_loop_1.yaml.j2 +++ b/parm/jcb-gdas/model/marine/marine_3dfgat_outer_loop_1.yaml.j2 @@ -1,7 +1,8 @@ - ninner: {{marine_ninner_1}} gradient norm reduction: {{marine_grad_red_1}} geometry: - mom6_input_nml: mom_input.nml + geom_grid_file: ./anl_geom/soca_gridspec.nc + mom6_input_nml: ./anl_geom/mom_input.nml fields metadata: {{marine_soca_files_path}}/fields_metadata.yaml diagnostics: departures: ombg diff --git a/parm/jcb-gdas/model/marine/marine_3dvar_outer_loop_1.yaml.j2 b/parm/jcb-gdas/model/marine/marine_3dvar_outer_loop_1.yaml.j2 deleted file mode 100644 index 2931f62a1..000000000 --- a/parm/jcb-gdas/model/marine/marine_3dvar_outer_loop_1.yaml.j2 +++ /dev/null @@ -1,7 +0,0 @@ -- ninner: {{marine_ninner_1}} - gradient norm reduction: {{marine_grad_red_1}} - geometry: - mom6_input_nml: mom_input.nml - fields metadata: {{marine_soca_files_path}}/fields_metadata.yaml - diagnostics: - departures: ombg diff --git a/parm/jcb-gdas/model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 b/parm/jcb-gdas/model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 index 5a5a1ac11..7edddcb8a 100644 --- a/parm/jcb-gdas/model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 +++ b/parm/jcb-gdas/model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 @@ -2,6 +2,7 @@ covariance model: hybrid components: - covariance: covariance model: SABER + change background resolution: true saber central block: saber block name: diffusion read: @@ -57,41 +58,44 @@ components: value: 1.00 - covariance: - covariance model: ensemble - members from template: - template: - read_from_file: 1 - date: '{{marine_stddev_time}}' - basename: ../ensdata/ - ocn_filename: 'ocn.pert.steric.%mem%.nc' - ice_filename: 'ens/ice.%mem%.nc' - state variables: - - sea_ice_area_fraction - - sea_ice_thickness - - sea_ice_snow_thickness - - sea_water_salinity - - sea_water_potential_temperature - #- eastward_sea_water_velocity - #- northward_sea_water_velocity - - sea_surface_height_above_geoid - pattern: '%mem%' - nmembers: '{{marine_number_ensemble_members}}' - localization: - localization method: SABER - saber central block: - saber block name: diffusion - read: - groups: - - variables: - - sea_water_potential_temperature - - sea_water_salinity - - sea_surface_height_above_geoid - - sea_ice_area_fraction - horizontal: - filepath: ./staticb/hz_ocean - vertical: - levels: {{marine_vt_levels}} - filepath: ./staticb/vt_ocean + covariance model: SABER + ensemble: + members from template: + template: + read_from_file: 1 + date: '{{marine_stddev_time}}' + basename: ../ensdata/ + ocn_filename: ocn.pert.%mem%.nc + ice_filename: ice.pert.%mem%.nc + state variables: + - sea_water_potential_temperature + - sea_water_salinity + #- eastward_sea_water_velocity + #- northward_sea_water_velocity + - sea_surface_height_above_geoid + - sea_ice_area_fraction + - sea_ice_thickness + - sea_ice_snow_thickness + pattern: '%mem%' + nmembers: '{{marine_number_ensemble_members}}' + saber central block: + saber block name: Ensemble + localization: + saber central block: + saber block name: diffusion + read: + groups: + - variables: + - sea_water_potential_temperature + - sea_water_salinity + - sea_surface_height_above_geoid + - sea_ice_area_fraction + multivariate strategy: duplicated + horizontal: + filepath: ./staticb/hz_ocean + vertical: + strategy: duplicated + weight: read_from_file: 3 basename: ./ diff --git a/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 b/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 index 2a41011d6..1ac58209f 100644 --- a/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 +++ b/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 @@ -1,4 +1,5 @@ covariance model: SABER +change background resolution: true saber central block: saber block name: diffusion read: diff --git a/parm/jcb-gdas/test/client_integration/gdas-marine-templates.yaml b/parm/jcb-gdas/test/client_integration/gdas-marine-templates.yaml index f258cc92a..238b4a83d 100644 --- a/parm/jcb-gdas/test/client_integration/gdas-marine-templates.yaml +++ b/parm/jcb-gdas/test/client_integration/gdas-marine-templates.yaml @@ -1,7 +1,6 @@ # This part is for testing only. Normally this would just show algorithm: # ----------------------------------------------------------------------------------- supported_algorithms: -- 3dvar - hofx3d - hofx4d - 3dfgat From 8c3514e6846b3d5f3d6d3ea54f1bebb719ed8975 Mon Sep 17 00:00:00 2001 From: Kevin Dougherty <69815622+kevindougherty-noaa@users.noreply.github.com> Date: Mon, 10 Feb 2025 10:42:32 -0500 Subject: [PATCH 086/199] Update stat anlaysis job templates and add snow template (#70) The following pull request updates the existing aero stats algorithms and adds a new snow template. These have been tested successfully for initial tests. These templates should provide as guides for future developments of other observation spaces. --------- Co-authored-by: Dan Holdaway <27729500+danholdaway@users.noreply.github.com> --- .../obstats/aero/viirs_n20_template.yaml.j2 | 14 +++--- .../obstats/aero/viirs_npp_template.yaml.j2 | 14 +++--- .../obstats/snow/ims_snow_template.yaml.j2 | 47 +++++++++++++++++++ 3 files changed, 61 insertions(+), 14 deletions(-) create mode 100644 parm/jcb-gdas/algorithm/obstats/snow/ims_snow_template.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/obstats/aero/viirs_n20_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/aero/viirs_n20_template.yaml.j2 index 9606dd922..50ad9c174 100644 --- a/parm/jcb-gdas/algorithm/obstats/aero/viirs_n20_template.yaml.j2 +++ b/parm/jcb-gdas/algorithm/obstats/aero/viirs_n20_template.yaml.j2 @@ -3,17 +3,17 @@ obsdatain: engine: type: H5File - obsfile: {{ aero_obsdatain_path }}/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc4 - simulated variables: {{ aero_obsdatatin_simulated_variables }} - observed variables: {{ aero_obsdatain_observed_variables }} - variables: {{ aero_variables }} - groups to process: {{ aero_file_groups }} - qc groups: {{ aero_file_qc_groups }} + obsfile: {{ aero_obsdatain_path }}/aero/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc4 + simulated variables: ['aerosolOpticalDepth'] + observed variables: ['aerosolOpticalDepth'] + variables: ['aerosolOpticalDepth'] + groups to process: ['ombg', 'oman'] + qc groups: ['EffectiveQC0', 'EffectiveQC1'] statistics to compute: ['mean', 'count', 'RMS'] output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_aod.nc" domains to process: - domain: - name: "nowhere" + name: "Global" first mask variable: latitude first mask range: [-180,-100] - domain: diff --git a/parm/jcb-gdas/algorithm/obstats/aero/viirs_npp_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/aero/viirs_npp_template.yaml.j2 index e0909bbbf..c46a47a99 100644 --- a/parm/jcb-gdas/algorithm/obstats/aero/viirs_npp_template.yaml.j2 +++ b/parm/jcb-gdas/algorithm/obstats/aero/viirs_npp_template.yaml.j2 @@ -3,17 +3,17 @@ obsdatain: engine: type: H5File - obsfile: {{ aero_obsdatain_path }}/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc4 - simulated variables: {{ aero_obsdatatin_simulated_variables }} - observed variables: {{ aero_obsdatain_observed_variables }} - variables: {{ aero_variables }} - groups to process: {{ aero_file_groups }} - qc groups: {{ aero_file_qc_groups }} + obsfile: {{ aero_obsdatain_path }}/aero/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc4 + simulated variables: ['aerosolOpticalDepth'] + observed variables: ['aerosolOpticalDepth'] + variables: ['aerosolOpticalDepth'] + groups to process: ['ombg', 'oman'] + qc groups: ['EffectiveQC0', 'EffectiveQC1'] statistics to compute: ['mean', 'count', 'RMS'] output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_aod.nc" domains to process: - domain: - name: "nowhere" + name: "Global" first mask variable: latitude first mask range: [-180,-100] - domain: diff --git a/parm/jcb-gdas/algorithm/obstats/snow/ims_snow_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/snow/ims_snow_template.yaml.j2 new file mode 100644 index 000000000..7e9880392 --- /dev/null +++ b/parm/jcb-gdas/algorithm/obstats/snow/ims_snow_template.yaml.j2 @@ -0,0 +1,47 @@ +- obs space: + name: ims_snow + obsdatain: + engine: + type: H5File + obsfile: {{ snow_obsdatain_path }}/snow/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc + simulated variables: ['totalSnowDepth'] + observed variables: ['totalSnowDepth'] + statistics to compute: ['mean', 'count', 'RMS'] + output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_snow.nc" + domains to process: + - domain: + name: "Global" + first mask variable: latitude + first mask range: [-180,-100] + - domain: + name: "SH" + first mask variable: latitude + first mask range: [-90,0] + - domain: + name: "NH" + first mask variable: latitude + first mask range: [0,90] + - domain: + name: "CONUS" + first mask variable: latitude + first mask range: [25,49] + second mask variable: longitude + second mask range: [-125,-66] + - domain: + name: "Europe" + first mask variable: latitude + first mask range: [35,70] + second mask variable: longitude + second mask range: [-11,38] + - domain: + name: "Africa" + first mask variable: latitude + first mask range: [-35,37] + second mask variable: longitude + second mask range: [-17,52] + - domain: + name: "Asia" + first mask variable: latitude + first mask range: [0,70] + second mask variable: longitude + second mask range: [38, 180] \ No newline at end of file From 54b7dac5bb4d4b345411e33fe2f25df8e9130107 Mon Sep 17 00:00:00 2001 From: Guillaume Vernieres Date: Fri, 21 Feb 2025 14:05:45 -0500 Subject: [PATCH 087/199] Remove the ens B from the marine hybrid B (#73) - fixes https://github.com/NOAA-EMC/GDASApp/issues/1491 --- .../algorithm/marine/soca_diagb.yaml.j2 | 5 +- ...d_error_hybrid_diffusion_diffusion.yaml.j2 | 3 + .../marine/sst_abi_g16_l3c.yaml.j2 | 63 +------------------ .../marine/sst_abi_g17_l3c.yaml.j2 | 63 +------------------ .../marine/sst_ahi_h08_l3c.yaml.j2 | 63 +------------------ .../marine/sst_avhrr_ma_l3u.yaml.j2 | 57 +---------------- .../marine/sst_avhrr_mb_l3u.yaml.j2 | 57 +---------------- .../marine/sst_avhrr_mc_l3u.yaml.j2 | 57 +---------------- .../observations/marine/sst_generic.yaml.j2 | 46 ++++++++++++++ .../marine/sst_viirs_n20_l3u.yaml.j2 | 57 +---------------- .../marine/sst_viirs_npp_l3u.yaml.j2 | 57 +---------------- 11 files changed, 60 insertions(+), 468 deletions(-) mode change 100644 => 120000 parm/jcb-gdas/observations/marine/sst_abi_g16_l3c.yaml.j2 mode change 100644 => 120000 parm/jcb-gdas/observations/marine/sst_abi_g17_l3c.yaml.j2 mode change 100644 => 120000 parm/jcb-gdas/observations/marine/sst_ahi_h08_l3c.yaml.j2 mode change 100644 => 120000 parm/jcb-gdas/observations/marine/sst_avhrr_ma_l3u.yaml.j2 mode change 100644 => 120000 parm/jcb-gdas/observations/marine/sst_avhrr_mb_l3u.yaml.j2 mode change 100644 => 120000 parm/jcb-gdas/observations/marine/sst_avhrr_mc_l3u.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 mode change 100644 => 120000 parm/jcb-gdas/observations/marine/sst_viirs_n20_l3u.yaml.j2 mode change 100644 => 120000 parm/jcb-gdas/observations/marine/sst_viirs_npp_l3u.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 index 1f7c0ebe5..17063d565 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 @@ -36,10 +36,11 @@ variables: - mom6_mld vertical e-folding scale: 500.0 +min efold depth ratio: 3.0 # keep the e-folding scale at most + # 1/(min efold depth ratio) of the local depth rescale: 1.0 # rescales the filtered std. dev. by "rescale" -min sst: 0.0 # Added to sst bkg. err. max ssh: 0.0 # Limits the amplitude of the unbalanced bkg err -min depth: 500.0 # zero out the bkg. error. at less than min depth +min depth: 50.0 # zero out the bkg. error. at less than min depth number of halo points: 4 number of neighbors: 16 diff --git a/parm/jcb-gdas/model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 b/parm/jcb-gdas/model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 index 7edddcb8a..30ff4218f 100644 --- a/parm/jcb-gdas/model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 +++ b/parm/jcb-gdas/model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 @@ -57,6 +57,7 @@ components: weight: value: 1.00 +{% if marine_ensemble_b | default(false) %} - covariance: covariance model: SABER ensemble: @@ -102,3 +103,5 @@ components: ocn_filename: 'ocean.ens_weights.nc' ice_filename: 'ice.ens_weights.nc' date: '{{marine_stddev_time}}' +{% endif %} + diff --git a/parm/jcb-gdas/observations/marine/sst_abi_g16_l3c.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_abi_g16_l3c.yaml.j2 deleted file mode 100644 index 2d912345c..000000000 --- a/parm/jcb-gdas/observations/marine/sst_abi_g16_l3c.yaml.j2 +++ /dev/null @@ -1,62 +0,0 @@ -- obs space: - name: {{observation_from_jcb}} - obsdatain: - engine: - type: H5File - obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" - obsdataout: - engine: - type: H5File - obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" - simulated variables: [seaSurfaceTemperature] - io pool: - max pool size: 1 - get values: - time interpolation: linear - obs operator: - name: Identity - observation alias file: obsop_name_map.yaml - obs error: - covariance model: diagonal - obs filters: - - filter: Domain Check - action: - name: passivate - where: - - variable: {name: ObsValue/seaSurfaceTemperature} - maxvalue: -4.0 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/sea_area_fraction} - minvalue: 0.9 - - filter: Bounds Check - minvalue: 1.0 - maxvalue: 41.0 - - filter: Background Check - absolute threshold: 5.0 - - filter: Domain Check - where: - - variable: {name: ObsError/seaSurfaceTemperature} - minvalue: 1.0e-8 - - filter: Domain Check - where: - - variable: { name: GeoVaLs/sea_ice_area_fraction} - maxvalue: 1.0e-5 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/sea_surface_temperature} - minvalue: -1.0 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 100e3 - - filter: Perform Action - action: - name: assign error - error function: - name: ObsFunction/LinearCombination - options: - variables: - - ObsError/seaSurfaceTemperature - coefs: - - 1.0 diff --git a/parm/jcb-gdas/observations/marine/sst_abi_g16_l3c.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_abi_g16_l3c.yaml.j2 new file mode 120000 index 000000000..5298b9a81 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/sst_abi_g16_l3c.yaml.j2 @@ -0,0 +1 @@ +sst_generic.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/sst_abi_g17_l3c.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_abi_g17_l3c.yaml.j2 deleted file mode 100644 index 2d912345c..000000000 --- a/parm/jcb-gdas/observations/marine/sst_abi_g17_l3c.yaml.j2 +++ /dev/null @@ -1,62 +0,0 @@ -- obs space: - name: {{observation_from_jcb}} - obsdatain: - engine: - type: H5File - obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" - obsdataout: - engine: - type: H5File - obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" - simulated variables: [seaSurfaceTemperature] - io pool: - max pool size: 1 - get values: - time interpolation: linear - obs operator: - name: Identity - observation alias file: obsop_name_map.yaml - obs error: - covariance model: diagonal - obs filters: - - filter: Domain Check - action: - name: passivate - where: - - variable: {name: ObsValue/seaSurfaceTemperature} - maxvalue: -4.0 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/sea_area_fraction} - minvalue: 0.9 - - filter: Bounds Check - minvalue: 1.0 - maxvalue: 41.0 - - filter: Background Check - absolute threshold: 5.0 - - filter: Domain Check - where: - - variable: {name: ObsError/seaSurfaceTemperature} - minvalue: 1.0e-8 - - filter: Domain Check - where: - - variable: { name: GeoVaLs/sea_ice_area_fraction} - maxvalue: 1.0e-5 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/sea_surface_temperature} - minvalue: -1.0 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 100e3 - - filter: Perform Action - action: - name: assign error - error function: - name: ObsFunction/LinearCombination - options: - variables: - - ObsError/seaSurfaceTemperature - coefs: - - 1.0 diff --git a/parm/jcb-gdas/observations/marine/sst_abi_g17_l3c.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_abi_g17_l3c.yaml.j2 new file mode 120000 index 000000000..5298b9a81 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/sst_abi_g17_l3c.yaml.j2 @@ -0,0 +1 @@ +sst_generic.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/sst_ahi_h08_l3c.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_ahi_h08_l3c.yaml.j2 deleted file mode 100644 index 2d912345c..000000000 --- a/parm/jcb-gdas/observations/marine/sst_ahi_h08_l3c.yaml.j2 +++ /dev/null @@ -1,62 +0,0 @@ -- obs space: - name: {{observation_from_jcb}} - obsdatain: - engine: - type: H5File - obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" - obsdataout: - engine: - type: H5File - obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" - simulated variables: [seaSurfaceTemperature] - io pool: - max pool size: 1 - get values: - time interpolation: linear - obs operator: - name: Identity - observation alias file: obsop_name_map.yaml - obs error: - covariance model: diagonal - obs filters: - - filter: Domain Check - action: - name: passivate - where: - - variable: {name: ObsValue/seaSurfaceTemperature} - maxvalue: -4.0 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/sea_area_fraction} - minvalue: 0.9 - - filter: Bounds Check - minvalue: 1.0 - maxvalue: 41.0 - - filter: Background Check - absolute threshold: 5.0 - - filter: Domain Check - where: - - variable: {name: ObsError/seaSurfaceTemperature} - minvalue: 1.0e-8 - - filter: Domain Check - where: - - variable: { name: GeoVaLs/sea_ice_area_fraction} - maxvalue: 1.0e-5 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/sea_surface_temperature} - minvalue: -1.0 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 100e3 - - filter: Perform Action - action: - name: assign error - error function: - name: ObsFunction/LinearCombination - options: - variables: - - ObsError/seaSurfaceTemperature - coefs: - - 1.0 diff --git a/parm/jcb-gdas/observations/marine/sst_ahi_h08_l3c.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_ahi_h08_l3c.yaml.j2 new file mode 120000 index 000000000..5298b9a81 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/sst_ahi_h08_l3c.yaml.j2 @@ -0,0 +1 @@ +sst_generic.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/sst_avhrr_ma_l3u.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_avhrr_ma_l3u.yaml.j2 deleted file mode 100644 index 63f847a92..000000000 --- a/parm/jcb-gdas/observations/marine/sst_avhrr_ma_l3u.yaml.j2 +++ /dev/null @@ -1,56 +0,0 @@ -- obs space: - name: {{observation_from_jcb}} - obsdatain: - engine: - type: H5File - obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" - obsdataout: - engine: - type: H5File - obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" - simulated variables: [seaSurfaceTemperature] - io pool: - max pool size: 1 - get values: - time interpolation: linear - obs operator: - name: Identity - observation alias file: obsop_name_map.yaml - obs error: - covariance model: diagonal - obs filters: - - filter: Domain Check - where: - - variable: {name: GeoVaLs/sea_area_fraction} - minvalue: 0.9 - - filter: Bounds Check - minvalue: 1.0 - maxvalue: 41.0 - - filter: Background Check - absolute threshold: 5.0 - - filter: Domain Check - where: - - variable: {name: ObsError/seaSurfaceTemperature} - minvalue: 1.0e-8 - - filter: Domain Check - where: - - variable: { name: GeoVaLs/sea_ice_area_fraction} - maxvalue: 1.0e-5 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/sea_surface_temperature} - minvalue: -1.0 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 100e3 - - filter: Perform Action - action: - name: assign error - error function: - name: ObsFunction/LinearCombination - options: - variables: - - ObsError/seaSurfaceTemperature - coefs: - - 1.0 diff --git a/parm/jcb-gdas/observations/marine/sst_avhrr_ma_l3u.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_avhrr_ma_l3u.yaml.j2 new file mode 120000 index 000000000..5298b9a81 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/sst_avhrr_ma_l3u.yaml.j2 @@ -0,0 +1 @@ +sst_generic.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/sst_avhrr_mb_l3u.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_avhrr_mb_l3u.yaml.j2 deleted file mode 100644 index 63f847a92..000000000 --- a/parm/jcb-gdas/observations/marine/sst_avhrr_mb_l3u.yaml.j2 +++ /dev/null @@ -1,56 +0,0 @@ -- obs space: - name: {{observation_from_jcb}} - obsdatain: - engine: - type: H5File - obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" - obsdataout: - engine: - type: H5File - obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" - simulated variables: [seaSurfaceTemperature] - io pool: - max pool size: 1 - get values: - time interpolation: linear - obs operator: - name: Identity - observation alias file: obsop_name_map.yaml - obs error: - covariance model: diagonal - obs filters: - - filter: Domain Check - where: - - variable: {name: GeoVaLs/sea_area_fraction} - minvalue: 0.9 - - filter: Bounds Check - minvalue: 1.0 - maxvalue: 41.0 - - filter: Background Check - absolute threshold: 5.0 - - filter: Domain Check - where: - - variable: {name: ObsError/seaSurfaceTemperature} - minvalue: 1.0e-8 - - filter: Domain Check - where: - - variable: { name: GeoVaLs/sea_ice_area_fraction} - maxvalue: 1.0e-5 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/sea_surface_temperature} - minvalue: -1.0 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 100e3 - - filter: Perform Action - action: - name: assign error - error function: - name: ObsFunction/LinearCombination - options: - variables: - - ObsError/seaSurfaceTemperature - coefs: - - 1.0 diff --git a/parm/jcb-gdas/observations/marine/sst_avhrr_mb_l3u.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_avhrr_mb_l3u.yaml.j2 new file mode 120000 index 000000000..5298b9a81 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/sst_avhrr_mb_l3u.yaml.j2 @@ -0,0 +1 @@ +sst_generic.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/sst_avhrr_mc_l3u.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_avhrr_mc_l3u.yaml.j2 deleted file mode 100644 index 63f847a92..000000000 --- a/parm/jcb-gdas/observations/marine/sst_avhrr_mc_l3u.yaml.j2 +++ /dev/null @@ -1,56 +0,0 @@ -- obs space: - name: {{observation_from_jcb}} - obsdatain: - engine: - type: H5File - obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" - obsdataout: - engine: - type: H5File - obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" - simulated variables: [seaSurfaceTemperature] - io pool: - max pool size: 1 - get values: - time interpolation: linear - obs operator: - name: Identity - observation alias file: obsop_name_map.yaml - obs error: - covariance model: diagonal - obs filters: - - filter: Domain Check - where: - - variable: {name: GeoVaLs/sea_area_fraction} - minvalue: 0.9 - - filter: Bounds Check - minvalue: 1.0 - maxvalue: 41.0 - - filter: Background Check - absolute threshold: 5.0 - - filter: Domain Check - where: - - variable: {name: ObsError/seaSurfaceTemperature} - minvalue: 1.0e-8 - - filter: Domain Check - where: - - variable: { name: GeoVaLs/sea_ice_area_fraction} - maxvalue: 1.0e-5 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/sea_surface_temperature} - minvalue: -1.0 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 100e3 - - filter: Perform Action - action: - name: assign error - error function: - name: ObsFunction/LinearCombination - options: - variables: - - ObsError/seaSurfaceTemperature - coefs: - - 1.0 diff --git a/parm/jcb-gdas/observations/marine/sst_avhrr_mc_l3u.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_avhrr_mc_l3u.yaml.j2 new file mode 120000 index 000000000..5298b9a81 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/sst_avhrr_mc_l3u.yaml.j2 @@ -0,0 +1 @@ +sst_generic.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 new file mode 100644 index 000000000..30754a0b5 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 @@ -0,0 +1,46 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaSurfaceTemperature] + io pool: + max pool size: 1 + get values: + time interpolation: linear + obs operator: + name: Identity + observation alias file: obsop_name_map.yaml + obs error: + covariance model: diagonal + obs filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + minvalue: 0.9 + - filter: Bounds Check + minvalue: -1.2 + maxvalue: 41.0 + - filter: Background Check + absolute threshold: 5.0 + - filter: Domain Check + where: + - variable: {name: ObsError/seaSurfaceTemperature} + minvalue: 1.0e-8 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_ice_area_fraction} + maxvalue: 1.0e-5 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_surface_temperature} + minvalue: -1.2 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/distance_from_coast} + minvalue: 50e3 diff --git a/parm/jcb-gdas/observations/marine/sst_viirs_n20_l3u.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_viirs_n20_l3u.yaml.j2 deleted file mode 100644 index 63f847a92..000000000 --- a/parm/jcb-gdas/observations/marine/sst_viirs_n20_l3u.yaml.j2 +++ /dev/null @@ -1,56 +0,0 @@ -- obs space: - name: {{observation_from_jcb}} - obsdatain: - engine: - type: H5File - obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" - obsdataout: - engine: - type: H5File - obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" - simulated variables: [seaSurfaceTemperature] - io pool: - max pool size: 1 - get values: - time interpolation: linear - obs operator: - name: Identity - observation alias file: obsop_name_map.yaml - obs error: - covariance model: diagonal - obs filters: - - filter: Domain Check - where: - - variable: {name: GeoVaLs/sea_area_fraction} - minvalue: 0.9 - - filter: Bounds Check - minvalue: 1.0 - maxvalue: 41.0 - - filter: Background Check - absolute threshold: 5.0 - - filter: Domain Check - where: - - variable: {name: ObsError/seaSurfaceTemperature} - minvalue: 1.0e-8 - - filter: Domain Check - where: - - variable: { name: GeoVaLs/sea_ice_area_fraction} - maxvalue: 1.0e-5 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/sea_surface_temperature} - minvalue: -1.0 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 100e3 - - filter: Perform Action - action: - name: assign error - error function: - name: ObsFunction/LinearCombination - options: - variables: - - ObsError/seaSurfaceTemperature - coefs: - - 1.0 diff --git a/parm/jcb-gdas/observations/marine/sst_viirs_n20_l3u.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_viirs_n20_l3u.yaml.j2 new file mode 120000 index 000000000..5298b9a81 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/sst_viirs_n20_l3u.yaml.j2 @@ -0,0 +1 @@ +sst_generic.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/sst_viirs_npp_l3u.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_viirs_npp_l3u.yaml.j2 deleted file mode 100644 index 63f847a92..000000000 --- a/parm/jcb-gdas/observations/marine/sst_viirs_npp_l3u.yaml.j2 +++ /dev/null @@ -1,56 +0,0 @@ -- obs space: - name: {{observation_from_jcb}} - obsdatain: - engine: - type: H5File - obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" - obsdataout: - engine: - type: H5File - obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" - simulated variables: [seaSurfaceTemperature] - io pool: - max pool size: 1 - get values: - time interpolation: linear - obs operator: - name: Identity - observation alias file: obsop_name_map.yaml - obs error: - covariance model: diagonal - obs filters: - - filter: Domain Check - where: - - variable: {name: GeoVaLs/sea_area_fraction} - minvalue: 0.9 - - filter: Bounds Check - minvalue: 1.0 - maxvalue: 41.0 - - filter: Background Check - absolute threshold: 5.0 - - filter: Domain Check - where: - - variable: {name: ObsError/seaSurfaceTemperature} - minvalue: 1.0e-8 - - filter: Domain Check - where: - - variable: { name: GeoVaLs/sea_ice_area_fraction} - maxvalue: 1.0e-5 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/sea_surface_temperature} - minvalue: -1.0 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 100e3 - - filter: Perform Action - action: - name: assign error - error function: - name: ObsFunction/LinearCombination - options: - variables: - - ObsError/seaSurfaceTemperature - coefs: - - 1.0 diff --git a/parm/jcb-gdas/observations/marine/sst_viirs_npp_l3u.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_viirs_npp_l3u.yaml.j2 new file mode 120000 index 000000000..5298b9a81 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/sst_viirs_npp_l3u.yaml.j2 @@ -0,0 +1 @@ +sst_generic.yaml.j2 \ No newline at end of file From ce41f5965a8a8a9805ac009acbb187c4cc17b1d1 Mon Sep 17 00:00:00 2001 From: Kevin Dougherty <69815622+kevindougherty-noaa@users.noreply.github.com> Date: Tue, 25 Feb 2025 11:25:43 -0500 Subject: [PATCH 088/199] Quick Filename Fix for Stats YAMLs (#80) Quick filename fix for the YAMLs needed for the analysis stats job in the g-w. --- parm/jcb-gdas/algorithm/obstats/aero/viirs_n20_template.yaml.j2 | 2 +- parm/jcb-gdas/algorithm/obstats/aero/viirs_npp_template.yaml.j2 | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/parm/jcb-gdas/algorithm/obstats/aero/viirs_n20_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/aero/viirs_n20_template.yaml.j2 index 50ad9c174..9994716ea 100644 --- a/parm/jcb-gdas/algorithm/obstats/aero/viirs_n20_template.yaml.j2 +++ b/parm/jcb-gdas/algorithm/obstats/aero/viirs_n20_template.yaml.j2 @@ -3,7 +3,7 @@ obsdatain: engine: type: H5File - obsfile: {{ aero_obsdatain_path }}/aero/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc4 + obsfile: {{ aero_obsdatain_path }}/aero/diag_{{ obspace }}_aod_{{ stat_current_cycle_YMDH }}.nc simulated variables: ['aerosolOpticalDepth'] observed variables: ['aerosolOpticalDepth'] variables: ['aerosolOpticalDepth'] diff --git a/parm/jcb-gdas/algorithm/obstats/aero/viirs_npp_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/aero/viirs_npp_template.yaml.j2 index c46a47a99..3c4c5d7f7 100644 --- a/parm/jcb-gdas/algorithm/obstats/aero/viirs_npp_template.yaml.j2 +++ b/parm/jcb-gdas/algorithm/obstats/aero/viirs_npp_template.yaml.j2 @@ -3,7 +3,7 @@ obsdatain: engine: type: H5File - obsfile: {{ aero_obsdatain_path }}/aero/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc4 + obsfile: {{ aero_obsdatain_path }}/aero/diag_{{ obspace }}_aod_{{ stat_current_cycle_YMDH }}.nc simulated variables: ['aerosolOpticalDepth'] observed variables: ['aerosolOpticalDepth'] variables: ['aerosolOpticalDepth'] From c045a4f99e247b3a3a6d119a78504d878602a904 Mon Sep 17 00:00:00 2001 From: Travis Elless <113720457+TravisElless-NOAA@users.noreply.github.com> Date: Wed, 26 Feb 2025 09:52:56 -0500 Subject: [PATCH 089/199] Update for missing snow files (#76) This PR adds missing file actions to the snow observation yamls and also makes an additional change to the obs staging yaml copy statement. These modifications are one step to ensuring the snow DA does not cause an operational failure if snow observation files are missing. References [GDASApp Issue 1462 ](https://github.com/NOAA-EMC/GDASApp/issues/1462) --- parm/jcb-gdas/algorithm/snow/snow_obs_staging.yaml.j2 | 2 +- parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 | 1 + parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 | 1 + parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 | 1 + parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 | 1 + 5 files changed, 5 insertions(+), 1 deletion(-) diff --git a/parm/jcb-gdas/algorithm/snow/snow_obs_staging.yaml.j2 b/parm/jcb-gdas/algorithm/snow/snow_obs_staging.yaml.j2 index a8d3dc863..778cfd1e5 100644 --- a/parm/jcb-gdas/algorithm/snow/snow_obs_staging.yaml.j2 +++ b/parm/jcb-gdas/algorithm/snow/snow_obs_staging.yaml.j2 @@ -1,6 +1,6 @@ mkdir: - '{{snow_obsdatain_path}}' -copy: +copy_opt: {% for observation_from_jcb in observations %} {% if use_observer(observation_from_jcb) %} {% if not observation_from_jcb == 'ims_snow' %} diff --git a/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 index f0fe2fd42..6298cc010 100644 --- a/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{snow_obsdatain_path}}/{{snow_obsdatain_prefix}}{{observation_from_jcb}}{{snow_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 index ea7a95655..b825e7937 100644 --- a/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{snow_obsdatain_path}}/{{snow_obsdatain_prefix}}{{observation_from_jcb}}.nc" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 b/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 index ce44417e9..31c8785be 100644 --- a/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 @@ -9,6 +9,7 @@ type: bufr obsfile: "{{snow_obsdatain_path}}/{{snow_obsdatain_prefix}}sfcsno.tm00.bufr_d" mapping file: "{{snow_obsdatain_path}}/bufr_sfcsno_mapping.yaml" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 index fe73587b0..c38da080e 100644 --- a/parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{snow_obsdatain_path}}/{{snow_obsdatain_prefix}}{{observation_from_jcb}}.nc4" + missing file action: warn obsdataout: engine: type: H5File From 6222c24d52cd5f62653519bb12b8e26f2ce6e547 Mon Sep 17 00:00:00 2001 From: JohnSteffen-NOAA <77806417+JohnSteffen-NOAA@users.noreply.github.com> Date: Wed, 26 Feb 2025 16:40:24 -0500 Subject: [PATCH 090/199] Add marine j2 yamls for tropical moorings and surface drifters (#81) This PR adds the marine j2 yamls to include tropical moorings and surface drifters for marine DA. --------- Co-authored-by: Guillaume Vernieres --- .../marine/insitu_profile_tropical.yaml.j2 | 33 +++++++++++++++++++ .../marine/insitu_surface_drifter.yaml.j2 | 33 +++++++++++++++++++ 2 files changed, 66 insertions(+) create mode 100644 parm/jcb-gdas/observations/marine/insitu_profile_tropical.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/insitu_surface_drifter.yaml.j2 diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_tropical.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_tropical.yaml.j2 new file mode 100644 index 000000000..8cdc4e166 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/insitu_profile_tropical.yaml.j2 @@ -0,0 +1,33 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [waterTemperature] + observed variables: [waterTemperature] + io pool: + max pool size: 1 + obs operator: + name: Composite + components: + - name: InsituTemperature + variables: + - name: waterTemperature + obs error: + covariance model: diagonal + obs filters: + - filter: Perform Action + action: + name: assign error + error function: + name: ObsFunction/LinearCombination + options: + variables: + - ObsError/waterTemperature + coefs: + - 1000.0 diff --git a/parm/jcb-gdas/observations/marine/insitu_surface_drifter.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_surface_drifter.yaml.j2 new file mode 100644 index 000000000..8cdc4e166 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/insitu_surface_drifter.yaml.j2 @@ -0,0 +1,33 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [waterTemperature] + observed variables: [waterTemperature] + io pool: + max pool size: 1 + obs operator: + name: Composite + components: + - name: InsituTemperature + variables: + - name: waterTemperature + obs error: + covariance model: diagonal + obs filters: + - filter: Perform Action + action: + name: assign error + error function: + name: ObsFunction/LinearCombination + options: + variables: + - ObsError/waterTemperature + coefs: + - 1000.0 From 7c69d39e8b80d8518b3f1f96e9b34a66acf9d529 Mon Sep 17 00:00:00 2001 From: AndrewEichmann-NOAA <58948505+AndrewEichmann-NOAA@users.noreply.github.com> Date: Thu, 27 Feb 2025 13:29:28 -0500 Subject: [PATCH 091/199] Add separate salt and temp configs for argo (#78) Adds separate salt and temperature obs space configuration files for Argo float observations. --- .../marine/insitu_salt_profile_argo.yaml.j2 | 178 ++++++++++++++++++ .../marine/insitu_temp_profile_argo.yaml.j2 | 173 +++++++++++++++++ 2 files changed, 351 insertions(+) create mode 100644 parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 diff --git a/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 new file mode 100644 index 000000000..8db7302fb --- /dev/null +++ b/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 @@ -0,0 +1,178 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsgrouping: + group variables: [latitude, longitude, dateTime] + sort variable: depth + sort group: MetaData + sort order: ascending + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [salinity] + observed variables: [salinity] + io pool: + max pool size: 1 + obs operator: + name: VertInterp + observation alias file: ./obsop_name_map.yaml + vertical coordinate: sea_water_depth + observation vertical coordinate: depth + interpolation method: linear + obs error: + covariance model: diagonal + + #------------------------------------------------------------------------------- + # START OF OBS FILTERS (work done by Kriti Bhargava) + # The QC filters used here are based on the document by IODE that can be found at + # https://cdn.ioos.noaa.gov/media/2017/12/recommendations_in_situ_data_real_time_qc.pdf + #------------------------------------------------------------------------------- + obs filters: + + # land check + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + minvalue: 0.5 + + + ## Filters for S: + #------------------------------------------------------------------------------- + #----------------------------------------------------------------------------- + ### Global range test + #----------------------------------------------------------------------------- + - filter: Bounds Check + filter variables: [{name: salinity}] + minvalue: 2.0 + maxvalue: 41.0 + + #----------------------------------------------------------------------------- + ### Regional range test + #----------------------------------------------------------------------------- + #### Red Sea + #----------------------------------------------------------------------------- + #### + #### the document linked here describes Red sea as the are between 10N, 40E; + #### 20N, 50E; 30N, 30E; 10N, 40E. But that would also include Gulf of Aden. + #### A more reasonable choice seemed to be a box that extends from 10 N to + #### 30 N and 30 E to 45 East . + + - filter: Bounds Check + filter variables: [{name: salinity}] + minvalue: 2.0 + maxvalue: 41.0 + where: + - variable: + name: MetaData/latitude + minvalue: 10 + maxvalue: 30 + - variable: + name: MetaData/longitude + minvalue: 30 + maxvalue: 45 + + #### Mediterranean Sea + #----------------------------------------------------------------------------- + ##### Area 1/3 for Mediterranean Sea + - filter: Bounds Check + filter variables: [{name: salinity}] + minvalue: 2.0 + maxvalue: 40.0 + where: + - variable: + name: MetaData/latitude + minvalue: 30 + maxvalue: 40 + - variable: + name: MetaData/longitude + minvalue: -6 + maxvalue: 40 + ##### Area 2/3 for Mediterranean Sea + - filter: Bounds Check + filter variables: [{name: salinity}] + minvalue: 2.0 + maxvalue: 40.0 + where: + - variable: + name: MetaData/latitude + minvalue: 40 + maxvalue: 41.5 + - variable: + name: MetaData/longitude + minvalue: 20 + maxvalue: 30 + ##### Area 3/3 for Mediterranean Sea + - filter: Bounds Check + filter variables: [{name: salinity}] + minvalue: 2.0 + maxvalue: 40.0 + where: + - variable: + name: MetaData/latitude + minvalue: 40 + maxvalue: 46 + - variable: + name: MetaData/longitude + minvalue: 0 + maxvalue: 20 + + + #### Northwestern shelves + #----------------------------------------------------------------------------- + - filter: Bounds Check + filter variables: [{name: salinity}] + minvalue: 0.0 + maxvalue: 37.0 + where: + - variable: + name: MetaData/latitude + minvalue: 50 + maxvalue: 60 + - variable: + name: MetaData/longitude + minvalue: -20 + maxvalue: 10 + + #### Southwestern shelves + #----------------------------------------------------------------------------- + - filter: Bounds Check + filter variables: [{name: salinity}] + minvalue: 0.0 + maxvalue: 38 + where: + - variable: + name: MetaData/latitude + minvalue: 25 + maxvalue: 50 + - variable: + name: MetaData/longitude + minvalue: -30 + maxvalue: 0 + + #### Arctic Ocean + #----------------------------------------------------------------------------- + - filter: Bounds Check + filter variables: [{name: salinity}] + minvalue: 2.0 + maxvalue: 40.0 + where: + - variable: + name: MetaData/latitude + minvalue: 60 + + - filter: Background Check + filter variables: [{name: salinity}] + threshold: 5.0 + absolute threshold: 5.0 + + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 diff --git a/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 new file mode 100644 index 000000000..98fac26ee --- /dev/null +++ b/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 @@ -0,0 +1,173 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsgrouping: + group variables: [latitude, longitude, dateTime] + sort variable: depth + sort group: MetaData + sort order: ascending + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [waterTemperature] + observed variables: [waterTemperature] + io pool: + max pool size: 1 + obs operator: + name: InsituTemperature + obs error: + covariance model: diagonal + + #------------------------------------------------------------------------------- + # START OF OBS FILTERS (work done by Kriti Bhargava) + # The QC filters used here are based on the document by IODE that can be found at + # https://cdn.ioos.noaa.gov/media/2017/12/recommendations_in_situ_data_real_time_qc.pdf + #------------------------------------------------------------------------------- + obs filters: + + # land check + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + minvalue: 0.5 + + #------------------------------------------------------------------------------- + ## Filters for T: + #------------------------------------------------------------------------------- + #------------------------------------------------------------------------------- + ### Global range test + #----------------------------------------------------------------------------- + - filter: Bounds Check + filter variables: [{name: waterTemperature}] + minvalue: -2.5 + maxvalue: 40.0 + + #----------------------------------------------------------------------------- + ### Regional range tests + #----------------------------------------------------------------------------- + + #### Red Sea + #----------------------------------------------------------------------------- + #### + #### the document linked here describes Red sea as the are between 10N, 40E; + #### 20N, 50E; 30N, 30E; 10N, 40E. But that would also include Gulf of Aden. + #### A more reasonable choice seemed to be a box that extends from 10 N to + #### 30 N and 30 E to 45 East . + + - filter: Bounds Check + filter variables: [{name: waterTemperature}] + minvalue: 21.7 + maxvalue: 40.0 + where: + - variable: + name: MetaData/latitude + minvalue: 10 + maxvalue: 30 + - variable: + name: MetaData/longitude + minvalue: 30 + maxvalue: 45 + + #### Mediterranean Sea + #----------------------------------------------------------------------------- + ##### Area 1/3 for Mediterranean Sea + - filter: Bounds Check + filter variables: [{name: waterTemperature}] + minvalue: 10.0 + maxvalue: 40.0 + where: + - variable: + name: MetaData/latitude + minvalue: 30 + maxvalue: 40 + - variable: + name: MetaData/longitude + minvalue: -6 + maxvalue: 40 + ##### Area 2/3 for Mediterranean Sea + - filter: Bounds Check + filter variables: [{name: waterTemperature}] + minvalue: 10.0 + maxvalue: 40.0 + where: + - variable: + name: MetaData/latitude + minvalue: 40 + maxvalue: 41.5 + - variable: + name: MetaData/longitude + minvalue: 20 + maxvalue: 30 + ##### Area 3/3 for Mediterranean Sea + - filter: Bounds Check + filter variables: [{name: waterTemperature}] + minvalue: 10.0 + maxvalue: 40.0 + where: + - variable: + name: MetaData/latitude + minvalue: 40 + maxvalue: 46 + - variable: + name: MetaData/longitude + minvalue: 0 + maxvalue: 20 + + #### Northwestern shelves + #----------------------------------------------------------------------------- + - filter: Bounds Check + filter variables: [{name: waterTemperature}] + minvalue: -2.0 + maxvalue: 24.0 + where: + - variable: + name: MetaData/latitude + minvalue: 50 + maxvalue: 60 + - variable: + name: MetaData/longitude + minvalue: -20 + maxvalue: 10 + #### Southwestern shelves + #----------------------------------------------------------------------------- + - filter: Bounds Check + filter variables: [{name: waterTemperature}] + minvalue: -2.0 + maxvalue: 30 + where: + - variable: + name: MetaData/latitude + minvalue: 25 + maxvalue: 50 + - variable: + name: MetaData/longitude + minvalue: -30 + maxvalue: 0 + + #### Arctic Ocean + #----------------------------------------------------------------------------- + - filter: Bounds Check + filter variables: [{name: waterTemperature}] + minvalue: -1.92 + maxvalue: 25.0 + where: + - variable: + name: MetaData/latitude + minvalue: 60 + + - filter: Background Check + filter variables: [{name: waterTemperature}] + threshold: 5.0 + absolute threshold: 5.0 + + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 From b20039126b24e21820b37f49e83ef913d14bf87a Mon Sep 17 00:00:00 2001 From: Jiarui Dong Date: Wed, 5 Mar 2025 11:14:12 -0500 Subject: [PATCH 092/199] Set up the reject list in observation chronicle for GTS synoptic snow depth data (#79) This PR takes the branch `feature/snow-example-chronicle` from @CoryMartin-NOAA and adds a initial list of the rejected station IDs in observation chronicle for GTS synoptic snow depth data by using https://github.com/NOAA-EMC/jcb/issues/17 This PR also adds/removes additional stations chronologically in the observation chronicle. This PR needs JCB updated to the commit hash `b114fe43` or later. --- .../snow/adpsfc_snow.yaml | 30 +++++++++++++++++++ .../observation_chronicle/snow/sfcsno.yaml | 30 +++++++++++++++++++ .../observations/snow/adpsfc_snow.yaml.j2 | 3 +- .../jcb-gdas/observations/snow/sfcsno.yaml.j2 | 3 +- 4 files changed, 62 insertions(+), 4 deletions(-) create mode 100644 parm/jcb-gdas/observation_chronicle/snow/adpsfc_snow.yaml create mode 100644 parm/jcb-gdas/observation_chronicle/snow/sfcsno.yaml diff --git a/parm/jcb-gdas/observation_chronicle/snow/adpsfc_snow.yaml b/parm/jcb-gdas/observation_chronicle/snow/adpsfc_snow.yaml new file mode 100644 index 000000000..c44170e8a --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/snow/adpsfc_snow.yaml @@ -0,0 +1,30 @@ +# Instrument metadata +# ------------------- +commissioned: 1970-01-01T00:00:00 + +observer_type: conventional # Type of chronicle to use + +window_option: max + +# observation type initial configuration +# -------------------------------- +stations_to_reject: [71120, 71397, 71621, 71727, 71816 # from North America + , 10863, 16179 # from Europe + , 40550, 40580, 40582, 40587, 40592 # Mid East + , 47005, 47008, 47014, 47016, 47020, 47022, 47025, 47028, 47031, 47035 # from North Korea + , 47037, 47039, 47041, 47046, 47050, 47052, 47055, 47058, 47060, 47061 + , 47065, 47067, 47068, 47069, 47070, 47069, 47070, 47075, 47090 + , 48698, 48830 # from Singapore + , 65250] # from Africa + +# Chronicle of changes for this observation type +# ---------------------------------------- +chronicles: + +- action_date: "2021-10-01T00:00:00" + justification: 'These stations are okay from this time' + remove_from_reject_list: [71120, 71397, 71727, 71816] + +- action_date: "2022-10-01T00:00:00" + justification: 'Add to reject these stations' + add_to_reject_list: [47095, 47098, 47101, 47102] diff --git a/parm/jcb-gdas/observation_chronicle/snow/sfcsno.yaml b/parm/jcb-gdas/observation_chronicle/snow/sfcsno.yaml new file mode 100644 index 000000000..c44170e8a --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/snow/sfcsno.yaml @@ -0,0 +1,30 @@ +# Instrument metadata +# ------------------- +commissioned: 1970-01-01T00:00:00 + +observer_type: conventional # Type of chronicle to use + +window_option: max + +# observation type initial configuration +# -------------------------------- +stations_to_reject: [71120, 71397, 71621, 71727, 71816 # from North America + , 10863, 16179 # from Europe + , 40550, 40580, 40582, 40587, 40592 # Mid East + , 47005, 47008, 47014, 47016, 47020, 47022, 47025, 47028, 47031, 47035 # from North Korea + , 47037, 47039, 47041, 47046, 47050, 47052, 47055, 47058, 47060, 47061 + , 47065, 47067, 47068, 47069, 47070, 47069, 47070, 47075, 47090 + , 48698, 48830 # from Singapore + , 65250] # from Africa + +# Chronicle of changes for this observation type +# ---------------------------------------- +chronicles: + +- action_date: "2021-10-01T00:00:00" + justification: 'These stations are okay from this time' + remove_from_reject_list: [71120, 71397, 71727, 71816] + +- action_date: "2022-10-01T00:00:00" + justification: 'Add to reject these stations' + add_to_reject_list: [47095, 47098, 47101, 47102] diff --git a/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 index 6298cc010..1ad679260 100644 --- a/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 @@ -76,8 +76,7 @@ where: - variable: name: MetaData/stationIdentification - is_in: [71120,71397,71621,71727,71816] - size where true: 5 + is_in: {{ get_conventional_rejected_stations(observation_from_jcb) }} obs post filters: - filter: Background Check # gross error check filter variables: diff --git a/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 b/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 index 31c8785be..b6151a7ad 100644 --- a/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 @@ -156,8 +156,7 @@ where: - variable: name: MetaData/stationIdentification - is_in: [71120,71397,71621,71727,71816] - size where true: 5 + is_in: {{ get_conventional_rejected_stations(observation_from_jcb) }} actions: - name: set flag: rejectlist From 986605cb47c4cf38c336c34445fe414b9e25f5da Mon Sep 17 00:00:00 2001 From: Yaping Wang <49168260+ypwang19@users.noreply.github.com> Date: Thu, 6 Mar 2025 10:47:09 -0500 Subject: [PATCH 093/199] Add aerosol bias correction to yaml templates (#83) The current aerosol DA templates does not have bias correction. In this PR, aerosol bias correction section is added to the viirs n20 observation templates. To use the Constant predictor with land-sea mask feature, sfc_data and slmsk variable will be added to the aerosol background template. --------- Co-authored-by: ypwang19 --- .../model/aero/aero_background.yaml.j2 | 3 ++- .../observations/aero/viirs_n20_aod.yaml.j2 | 18 ++++++++++++++++++ 2 files changed, 20 insertions(+), 1 deletion(-) diff --git a/parm/jcb-gdas/model/aero/aero_background.yaml.j2 b/parm/jcb-gdas/model/aero/aero_background.yaml.j2 index 012c50db5..8d9c9e8bd 100644 --- a/parm/jcb-gdas/model/aero/aero_background.yaml.j2 +++ b/parm/jcb-gdas/model/aero/aero_background.yaml.j2 @@ -4,7 +4,8 @@ datetime: '{{ aero_background_time_iso }}' filename is datetime templated: true filename_core: '%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc' filename_trcr: '%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc' +filename_sfcd: '%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc' filename_cplr: '%yyyy%mm%dd.%hh%MM%ss.coupler.res' state variables: [t,delp,sphum,so4,bc1,bc2,oc1,oc2, dust1,dust2,dust3,dust4,dust5, - seas1,seas2,seas3,seas4] + seas1,seas2,seas3,seas4,slmsk,sheleg] diff --git a/parm/jcb-gdas/observations/aero/viirs_n20_aod.yaml.j2 b/parm/jcb-gdas/observations/aero/viirs_n20_aod.yaml.j2 index 5ef6aca35..26e34f643 100644 --- a/parm/jcb-gdas/observations/aero/viirs_n20_aod.yaml.j2 +++ b/parm/jcb-gdas/observations/aero/viirs_n20_aod.yaml.j2 @@ -35,6 +35,24 @@ # Observation Bias Correction (VarBC) # ----------------------------------- + obs bias: + input file: "{{aero_obsbiasin_path}}/{{aero_obsbiasin_prefix}}{{observation_from_jcb}}{{aero_obsbiasin_suffix}}" + output file: "{{aero_obsbiasout_path}}/{{aero_obsbiasout_prefix}}{{observation_from_jcb}}{{aero_obsbiasout_suffix}}" + variational bc: + predictors: + - name: constant + surface: land sea mask + covariance: + minimal required obs number: 20 + variance range: [1.0e-6, 10.0] + step size: 1.0e-4 + largest analysis variance: 10000.0 + prior: + input file: "{{aero_obsbiasin_path}}/{{aero_obsbiascovin_prefix}}{{observation_from_jcb}}{{aero_obsbiascovin_suffix}}" + inflation: + ratio: 1.1 + ratio for small dataset: 2.0 + output file: "{{aero_obsbiasout_path}}/{{aero_obsbiascovout_prefix}}{{observation_from_jcb}}{{aero_obsbiascovout_suffix}}" # Observation Filters (QC) # ------------------------ From 6eaed2339ff863c689503bed15892b3fa997bfd4 Mon Sep 17 00:00:00 2001 From: DavidNew-NOAA <134300700+DavidNew-NOAA@users.noreply.github.com> Date: Mon, 10 Mar 2025 09:15:25 -0400 Subject: [PATCH 094/199] JEDI-based ensemble recentering and analysis calculation and native grid increments (#72) This PR is a companion to GDASApp PR [#1488](https://github.com/NOAA-EMC/GDASApp/pull/1488) and Global Workflow PR [#3312](https://github.com/NOAA-EMC/global-workflow/pull/3312). It creates YAMLs for the JEDI-based ensemble recentering application and the `convertstate` applications used for analysis calculation. It also alters certain YAMLs to implement native-grid increments in the Global Workflow. Some miscellaneous changes to the naming convention of increments and backgrounds in run directories are also implemented for overall consistency and simplicity across JEDI-based Global Workflow jobs. --------- Co-authored-by: danholdaway Co-authored-by: Jiarui Dong Co-authored-by: Cory Martin --- .../algorithm/aero/aero_addincrement.yaml.j2 | 43 +++++++++++ .../atmosphere/atm_addincrement.yaml.j2 | 41 +++++++++++ .../fv3jedi_correction_increment.yaml.j2 | 73 +++++++++++++++++++ .../fv3jedi_ensemble_recenter.yaml.j2 | 47 ++++++++++++ .../atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 | 12 +-- .../fv3jedi_fv3inc_variational.yaml.j2 | 26 ++++--- .../algorithm/snow/snow_addincrement.yaml.j2 | 43 +++++++++++ ...phere_final_increment_cubed_sphere.yaml.j2 | 8 +- ...ere_final_increment_fms_nonrestart.yaml.j2 | 18 +++++ ...t_ensemble_increments_cubed_sphere.yaml.j2 | 2 +- ...ensemble_increments_fms_nonrestart.yaml.j2 | 4 + ...here_posterior_output_cubed_sphere.yaml.j2 | 5 ++ .../gdas-atmosphere-templates.yaml | 3 + 13 files changed, 304 insertions(+), 21 deletions(-) create mode 100644 parm/jcb-gdas/algorithm/aero/aero_addincrement.yaml.j2 create mode 100644 parm/jcb-gdas/algorithm/atmosphere/atm_addincrement.yaml.j2 create mode 100644 parm/jcb-gdas/algorithm/atmosphere/fv3jedi_correction_increment.yaml.j2 create mode 100644 parm/jcb-gdas/algorithm/atmosphere/fv3jedi_ensemble_recenter.yaml.j2 create mode 100644 parm/jcb-gdas/algorithm/snow/snow_addincrement.yaml.j2 create mode 100644 parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_fms_nonrestart.yaml.j2 create mode 100644 parm/jcb-gdas/model/atmosphere/atmosphere_output_ensemble_increments_fms_nonrestart.yaml.j2 create mode 100644 parm/jcb-gdas/model/atmosphere/atmosphere_posterior_output_cubed_sphere.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/aero/aero_addincrement.yaml.j2 b/parm/jcb-gdas/algorithm/aero/aero_addincrement.yaml.j2 new file mode 100644 index 000000000..b950f5cb1 --- /dev/null +++ b/parm/jcb-gdas/algorithm/aero/aero_addincrement.yaml.j2 @@ -0,0 +1,43 @@ +state geometry: + fms initialization: + namelist filename: ./fv3jedi/fmsmpp.nml + field table filename: ./fv3jedi/field_table + akbk: ./fv3jedi/akbk.nc4 + layout: + - {{ aero_layout_x }} + - {{ aero_layout_y }} + npx: {{ aero_npx_ges }} + npy: {{ aero_npy_ges }} + npz: {{ aero_npz_ges }} + field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_history.yaml +increment geometry: + fms initialization: + namelist filename: ./fv3jedi/fmsmpp.nml + field table filename: ./fv3jedi/field_table + akbk: ./fv3jedi/akbk.nc4 + layout: + - {{ aero_layout_x }} + - {{ aero_layout_y }} + npx: {{ aero_npx_ges }} + npy: {{ aero_npy_ges }} + npz: {{ aero_npz_ges }} + field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_restart.yaml +state: + datetime: {{ aero_background_time_iso }} + state variables: {{ analysis_variables }} + filetype: cube sphere history + provider: ufs + datapath: ./ + filename: {{ aero_variational_history_prefix }}cubed_sphere_grid_atmf006.nc +increment: + added variables: {{ analysis_variables }} + filetype: fms restart + datapath: ./ + filename is datetime templated: true + filename_trcr: '%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc' + prefix: aeroinc +output: + filetype: auxgrid + gridtype: gaussian + datapath: ./ + filename: aeroanl. diff --git a/parm/jcb-gdas/algorithm/atmosphere/atm_addincrement.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/atm_addincrement.yaml.j2 new file mode 100644 index 000000000..0f0d2ed77 --- /dev/null +++ b/parm/jcb-gdas/algorithm/atmosphere/atm_addincrement.yaml.j2 @@ -0,0 +1,41 @@ +state geometry: + fms initialization: + namelist filename: ./fv3jedi/fmsmpp.nml + field table filename: ./fv3jedi/field_table + akbk: ./fv3jedi/akbk.nc4 + layout: + - {{ atmosphere_layout_x }} + - {{ atmosphere_layout_y }} + npx: {{ atmosphere_npx_ges }} + npy: {{ atmosphere_npy_ges }} + npz: {{ atmosphere_npz_ges }} + field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_history.yaml +increment geometry: + fms initialization: + namelist filename: ./fv3jedi/fmsmpp.nml + field table filename: ./fv3jedi/field_table + akbk: ./fv3jedi/akbk.nc4 + layout: + - {{ atmosphere_layout_x }} + - {{ atmosphere_layout_y }} + npx: {{ atmosphere_npx_ges }} + npy: {{ atmosphere_npy_ges }} + npz: {{ atmosphere_npz_ges }} + field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_fv3inc.yaml +state: + datetime: {{ atmosphere_background_time_iso }} + state variables: [ua,va,t,sphum,ice_wat,liq_wat,o3mr,delp,delz] + filetype: cube sphere history + provider: ufs + filename: {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_atmf006.nc + datapath: ./ +increment: + added variables: [ua,va,t,sphum,ice_wat,liq_wat,o3mr,delp,delz] + filetype: fms restart + is restart: false + filename_nonrestart: {{ atmosphere_variational_analysis_prefix }}cubed_sphere_grid_atminc.nc + datapath: ./ +output: + filetype: auxgrid + gridtype: gaussian + filename: atmanl. diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_correction_increment.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_correction_increment.yaml.j2 new file mode 100644 index 000000000..07a9b2119 --- /dev/null +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_correction_increment.yaml.j2 @@ -0,0 +1,73 @@ +increment variables: [liq_wat,delz,delp,ice_wat,o3mr,sphum,t,ua,va] +deterministic background geometry: + fms initialization: + namelist filename: ./fv3jedi/fmsmpp.nml + field table filename: ./fv3jedi/field_table + akbk: ./fv3jedi/akbk.nc4 + layout: + - {{ atmosphere_layout_x }} + - {{ atmosphere_layout_y }} + npx: {{ atmosphere_npx_ges }} + npy: {{ atmosphere_npy_ges }} + npz: {{ atmosphere_npz_ges }} + field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_history.yaml +variational increment geometry: + fms initialization: + namelist filename: ./fv3jedi/fmsmpp.nml + field table filename: ./fv3jedi/field_table + akbk: ./fv3jedi/akbk.nc4 + layout: + - {{ atmosphere_layout_x }} + - {{ atmosphere_layout_y }} + npx: {{ atmosphere_npx_ges }} + npy: {{ atmosphere_npy_ges }} + npz: {{ atmosphere_npz_ges }} + field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_fv3inc.yaml +ensemble mean analysis geometry: + fms initialization: + namelist filename: ./fv3jedi/fmsmpp.nml + field table filename: ./fv3jedi/field_table + akbk: ./fv3jedi/akbk.nc4 + layout: + - {{ atmosphere_layout_x }} + - {{ atmosphere_layout_y }} + npx: {{ atmosphere_npx_anl }} + npy: {{ atmosphere_npy_anl }} + npz: {{ atmosphere_npz_anl }} + field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_history.yaml +correction increment geometry: + fms initialization: + namelist filename: ./fv3jedi/fmsmpp.nml + field table filename: ./fv3jedi/field_table + akbk: ./fv3jedi/akbk.nc4 + layout: + - {{ atmosphere_layout_x }} + - {{ atmosphere_layout_y }} + npx: {{ atmosphere_npx_anl }} + npy: {{ atmosphere_npy_anl }} + npz: {{ atmosphere_npz_anl }} + field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_fv3inc.yaml +forecast hours: +{% for ihour in range( atmosphere_iau_hours | length ) %} +- datetime: {{ atmosphere_iau_times_iso[ihour] }} + deterministic background: + filetype: cube sphere history + provider: ufs + datapath: ./ + filename: {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_atmf{{ '%03d' % atmosphere_iau_hours[ihour] }}.nc + variational increment: + filetype: fms restart + is restart: false + datapath: ./ + filename_nonrestart: {{ atmosphere_variational_analysis_prefix }}cubed_sphere_grid_atmi{{ '%03d' % atmosphere_iau_hours[ihour] }}.nc + ensemble mean analysis: + filetype: cube sphere history + provider: ufs + datapath: ./ + filename: {{ atmosphere_ensemble_analysis_prefix }}cubed_sphere_grid_atma{{ '%03d' % atmosphere_iau_hours[ihour] }}.ensmean.nc + correction increment: + filetype: cube sphere history + provider: ufs + datapath: ./ + filename: {{ atmosphere_ensemble_analysis_prefix }}cubed_sphere_grid_catmi{{ '%03d' % atmosphere_iau_hours[ihour] }}.nc +{% endfor %} diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_ensemble_recenter.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_ensemble_recenter.yaml.j2 new file mode 100644 index 000000000..28382b33a --- /dev/null +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_ensemble_recenter.yaml.j2 @@ -0,0 +1,47 @@ +members: +{% for imember in range(atmosphere_number_ensemble_members) %} +{% for ihour in range( atmosphere_iau_hours | length ) %} +- state geometry: + fms initialization: + namelist filename: ./fv3jedi/fmsmpp.nml + field table filename: ./fv3jedi/field_table + akbk: ./fv3jedi/akbk.nc4 + layout: + - {{ atmosphere_layout_x }} + - {{ atmosphere_layout_y }} + npx: {{ atmosphere_npx_anl }} + npy: {{ atmosphere_npy_anl }} + npz: {{ atmosphere_npz_anl }} + field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_fv3inc.yaml + increment geometry: + fms initialization: + namelist filename: ./fv3jedi/fmsmpp.nml + field table filename: ./fv3jedi/field_table + akbk: ./fv3jedi/akbk.nc4 + layout: + - {{ atmosphere_layout_x }} + - {{ atmosphere_layout_y }} + npx: {{ atmosphere_npx_anl }} + npy: {{ atmosphere_npy_anl }} + npz: {{ atmosphere_npz_anl }} + field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_history.yaml + state: + datetime: {{ atmosphere_iau_times_iso[ihour] }} + state variables: [ua,va,t,sphum,ice_wat,liq_wat,o3mr,delp,delz] + filetype: cube sphere history + provider: ufs + datapath: ./ + filename: {{ atmosphere_ensemble_analysis_prefix }}cubed_sphere_grid_catmi{{ '%03d' % atmosphere_iau_hours[ihour] }}.nc + increment: + added variables: [ua,va,t,sphum,ice_wat,liq_wat,o3mr,delp,delz] + filetype: cube sphere history + provider: ufs + datapath: ./mem{{ '%0{}d'.format(3)|format(imember+1) }} + filename: {{ atmosphere_ensemble_analysis_prefix }}cubed_sphere_grid_atmi{{ '%03d' % atmosphere_iau_hours[ihour] }}.nc + output: + filetype: fms restart + is restart: false + datapath: ./mem{{ '%0{}d'.format(3)|format(imember+1) }} + filename_nonrestart: {{ atmosphere_ensemble_analysis_prefix }}cubed_sphere_grid_ratmi{{ '%03d' % atmosphere_iau_hours[ihour] }}.nc +{% endfor %} +{% endfor %} diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 index 576a133eb..cac17a0fe 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 @@ -39,7 +39,7 @@ fv3 increment geometry: npx: {{ atmosphere_npx_ges }} npy: {{ atmosphere_npy_ges }} npz: {{ atmosphere_npz_ges }} - field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_fv3inc.yaml + field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_history.yaml members from template: template: background input: @@ -53,12 +53,14 @@ members from template: state variables: *bkgvars jedi increment input: filetype: cube sphere history - filename: ./anl/mem%mem%/atminc.%yyyy%mm%dd.%hh%MM%ssz.nc4 provider: ufs + datapath: ./anl/mem%mem% + filename: cubed_sphere_grid_atminc.jedi.nc fv3 increment output: - filetype: auxgrid - gridtype: gaussian - filename: ./anl/mem%mem%/atminc. + filetype: cube sphere history + provider: ufs + datapath: ./anl/mem%mem% + filename: {{ atmosphere_ensemble_analysis_prefix }}cubed_sphere_grid_atminc.nc pattern: '%mem%' nmembers: {{ atmosphere_number_ensemble_members }} zero padding: 3 diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 index dd60ebb75..91c47f0df 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 @@ -24,9 +24,9 @@ jedi increment geometry: layout: - {{ atmosphere_layout_x }} - {{ atmosphere_layout_y }} - npx: {{ atmosphere_npx_anl }} - npy: {{ atmosphere_npy_anl }} - npz: {{ atmosphere_npz_anl }} + npx: {{ atmosphere_npx_ges }} + npy: {{ atmosphere_npy_ges }} + npz: {{ atmosphere_npz_ges }} field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_history.yaml fv3 increment geometry: fms initialization: @@ -36,9 +36,9 @@ fv3 increment geometry: layout: - {{ atmosphere_layout_x }} - {{ atmosphere_layout_y }} - npx: {{ atmosphere_npx_anl }} - npy: {{ atmosphere_npy_anl }} - npz: {{ atmosphere_npz_anl }} + npx: {{ atmosphere_npx_ges }} + npy: {{ atmosphere_npy_ges }} + npz: {{ atmosphere_npz_ges }} field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_fv3inc.yaml members: - background input: @@ -48,16 +48,20 @@ members: ufs soil nlev: 4 ufs fields split by level: [smc,stc] datetime: "{{ atmosphere_background_time_iso }}" - filenames: [ {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_atmf006.nc, {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_sfcf006.nc ] + filenames: + - {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_atmf006.nc + - {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_sfcf006.nc state variables: *bkgvars jedi increment input: filetype: cube sphere history - filename: ./anl/atminc.%yyyy%mm%dd.%hh%MM%ssz.nc4 provider: ufs + datapath: ./anl + filename: cubed_sphere_grid_atminc.jedi.nc fv3 increment output: - filetype: auxgrid - gridtype: gaussian - filename: ./anl/atminc. + filetype: fms restart + is restart: false + datapath: ./anl + filename_nonrestart: {{ atmosphere_variational_analysis_prefix }}cubed_sphere_grid_atminc.nc # Optionally test the application {% if do_testing | default(false) %} diff --git a/parm/jcb-gdas/algorithm/snow/snow_addincrement.yaml.j2 b/parm/jcb-gdas/algorithm/snow/snow_addincrement.yaml.j2 new file mode 100644 index 000000000..96288640b --- /dev/null +++ b/parm/jcb-gdas/algorithm/snow/snow_addincrement.yaml.j2 @@ -0,0 +1,43 @@ +state geometry: + fms initialization: + namelist filename: ./fv3jedi/fmsmpp.nml + field table filename: ./fv3jedi/field_table + akbk: ./fv3jedi/akbk.nc4 + layout: + - {{ snow_layout_x }} + - {{ snow_layout_y }} + npx: {{ snow_npx_ges }} + npy: {{ snow_npy_ges }} + npz: {{ snow_npz_ges }} + field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_history.yaml +increment geometry: + fms initialization: + namelist filename: ./fv3jedi/fmsmpp.nml + field table filename: ./fv3jedi/field_table + akbk: ./fv3jedi/akbk.nc4 + layout: + - {{ snow_layout_x }} + - {{ snow_layout_y }} + npx: {{ snow_npx_ges }} + npy: {{ snow_npy_ges }} + npz: {{ snow_npz_ges }} + field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_restart.yaml +state: + datetime: {{ snow_background_time_iso }} + state variables: {{ analysis_variables }} + filetype: cube sphere history + provider: ufs + datapath: ./ + filename: {{ snow_variational_history_prefix }}cubed_sphere_grid_sfcf006.nc +increment: + added variables: {{ analysis_variables }} + filetype: fms restart + datapath: ./ + filename is datetime templated: true + filename_sfcd: '%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc' + prefix: snowinc +output: + filetype: auxgrid + gridtype: gaussian + datapath: ./ + filename: snowanl. diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_cubed_sphere.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_cubed_sphere.yaml.j2 index 7d9f7e7e0..e49a7daf0 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_cubed_sphere.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_cubed_sphere.yaml.j2 @@ -1,7 +1,7 @@ output: state component: filetype: cube sphere history - filename: ./anl/atminc.%yyyy%mm%dd.%hh%MM%ssz.nc4 + filename: ./anl/cubed_sphere_grid_atminc.jedi.nc provider: ufs fields to write: [ugrd,vgrd,tmp,pressfc,spfh,icmr,clwmr,o3mr] geometry: @@ -12,7 +12,7 @@ geometry: layout: - {{atmosphere_layout_x}} - {{atmosphere_layout_y}} - npx: {{atmosphere_npx_anl}} - npy: {{atmosphere_npy_anl}} - npz: {{atmosphere_npz_anl}} + npx: {{atmosphere_npx_ges}} + npy: {{atmosphere_npy_ges}} + npz: {{atmosphere_npz_ges}} field metadata override: "{{atmosphere_fv3jedi_files_path}}/fv3jedi_fieldmetadata_history.yaml" diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_fms_nonrestart.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_fms_nonrestart.yaml.j2 new file mode 100644 index 000000000..efac27abe --- /dev/null +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_fms_nonrestart.yaml.j2 @@ -0,0 +1,18 @@ +output: + state component: + filetype: fms restart + is restart: false + filename_nonrestart: ./anl/cubed_sphere_grid_atminc.jedi.nc + fields to write: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] +geometry: + fms initialization: + namelist filename: "{{atmosphere_fv3jedi_files_path}}/fmsmpp.nml" + field table filename: "{{atmosphere_fv3jedi_files_path}}/field_table" + akbk: "{{atmosphere_fv3jedi_files_path}}/akbk.nc4" + layout: + - {{atmosphere_layout_x}} + - {{atmosphere_layout_y}} + npx: {{atmosphere_npx_ges}} + npy: {{atmosphere_npy_ges}} + npz: {{atmosphere_npz_ges}} + field metadata override: "{{atmosphere_fv3jedi_files_path}}/fv3jedi_fieldmetadata_fv3inc.yaml" diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_output_ensemble_increments_cubed_sphere.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_output_ensemble_increments_cubed_sphere.yaml.j2 index cf655f1c7..8d39113bd 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_output_ensemble_increments_cubed_sphere.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_output_ensemble_increments_cubed_sphere.yaml.j2 @@ -1,3 +1,3 @@ filetype: cube sphere history provider: ufs -filename: ./anl/mem%{member}%/atminc.%yyyy%mm%dd.%hh%MM%ssz.nc4 +filename: ./anl/mem%{member}%/cubed_sphere_grid_atminc.jedi.nc diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_output_ensemble_increments_fms_nonrestart.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_output_ensemble_increments_fms_nonrestart.yaml.j2 new file mode 100644 index 000000000..ff021176f --- /dev/null +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_output_ensemble_increments_fms_nonrestart.yaml.j2 @@ -0,0 +1,4 @@ +filetype: fms restart +is restart: false +datapath: ./anl/mem%{member}% +filename_nonrestart: cubed_sphere_grid_atminc.jedi.nc diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_posterior_output_cubed_sphere.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_posterior_output_cubed_sphere.yaml.j2 new file mode 100644 index 000000000..aa72fc7e3 --- /dev/null +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_posterior_output_cubed_sphere.yaml.j2 @@ -0,0 +1,5 @@ +filetype: cube sphere history +provider: ufs +ufs soil nlev: 4 +ufs fields split by level: [smc,stc] +filename: ./anl/{{ atmosphere_ensemble_analysis_prefix }}cubed_sphere_grid_atmanl.ensmean.nc diff --git a/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml b/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml index 4f66e4dfb..006567743 100644 --- a/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml +++ b/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml @@ -63,6 +63,9 @@ atmosphere_background_ensemble_path: "DATA/ens/mem%mem%" atmosphere_variational_history_prefix: "bkg_" atmosphere_ensemble_history_prefix: "ens_" +atmosphere_variational_analysis_prefix: "anl_" +atmosphere_ensemble_analysis_prefix: "anl_" + atmosphere_background_time_iso: '2024-02-02T00:00:00Z' # Background error From 917ff871f197746ee6b0d0f73b63b9e90ef20d5d Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Mon, 10 Mar 2025 13:20:34 +0000 Subject: [PATCH 095/199] Revert "Add aerosol bias correction to yaml templates" (#84) Reverts NOAA-EMC/jcb-gdas#83 temporarily until BC files are ready in the workflow and we can do a coordinated merge --- .../model/aero/aero_background.yaml.j2 | 3 +-- .../observations/aero/viirs_n20_aod.yaml.j2 | 18 ------------------ 2 files changed, 1 insertion(+), 20 deletions(-) diff --git a/parm/jcb-gdas/model/aero/aero_background.yaml.j2 b/parm/jcb-gdas/model/aero/aero_background.yaml.j2 index 8d9c9e8bd..012c50db5 100644 --- a/parm/jcb-gdas/model/aero/aero_background.yaml.j2 +++ b/parm/jcb-gdas/model/aero/aero_background.yaml.j2 @@ -4,8 +4,7 @@ datetime: '{{ aero_background_time_iso }}' filename is datetime templated: true filename_core: '%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc' filename_trcr: '%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc' -filename_sfcd: '%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc' filename_cplr: '%yyyy%mm%dd.%hh%MM%ss.coupler.res' state variables: [t,delp,sphum,so4,bc1,bc2,oc1,oc2, dust1,dust2,dust3,dust4,dust5, - seas1,seas2,seas3,seas4,slmsk,sheleg] + seas1,seas2,seas3,seas4] diff --git a/parm/jcb-gdas/observations/aero/viirs_n20_aod.yaml.j2 b/parm/jcb-gdas/observations/aero/viirs_n20_aod.yaml.j2 index 26e34f643..5ef6aca35 100644 --- a/parm/jcb-gdas/observations/aero/viirs_n20_aod.yaml.j2 +++ b/parm/jcb-gdas/observations/aero/viirs_n20_aod.yaml.j2 @@ -35,24 +35,6 @@ # Observation Bias Correction (VarBC) # ----------------------------------- - obs bias: - input file: "{{aero_obsbiasin_path}}/{{aero_obsbiasin_prefix}}{{observation_from_jcb}}{{aero_obsbiasin_suffix}}" - output file: "{{aero_obsbiasout_path}}/{{aero_obsbiasout_prefix}}{{observation_from_jcb}}{{aero_obsbiasout_suffix}}" - variational bc: - predictors: - - name: constant - surface: land sea mask - covariance: - minimal required obs number: 20 - variance range: [1.0e-6, 10.0] - step size: 1.0e-4 - largest analysis variance: 10000.0 - prior: - input file: "{{aero_obsbiasin_path}}/{{aero_obsbiascovin_prefix}}{{observation_from_jcb}}{{aero_obsbiascovin_suffix}}" - inflation: - ratio: 1.1 - ratio for small dataset: 2.0 - output file: "{{aero_obsbiasout_path}}/{{aero_obsbiascovout_prefix}}{{observation_from_jcb}}{{aero_obsbiascovout_suffix}}" # Observation Filters (QC) # ------------------------ From 5da0ef55fca45570e7918566adff9ce1d98f7d80 Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Fri, 14 Mar 2025 20:09:14 +0000 Subject: [PATCH 096/199] Add sample conventional surface pressure template for generating summary stats (#85) This PR adds a template for a conventional surface pressure obs space for `ioda-stats` as an example of how to do this for all atmospheric obs spaces. --- .../obstats/aero/viirs_n20_template.yaml.j2 | 4 -- .../obstats/aero/viirs_npp_template.yaml.j2 | 6 +-- .../atmos/conventional_ps_template.yaml.j2 | 43 +++++++++++++++++++ .../obstats/snow/ims_snow_template.yaml.j2 | 6 +-- 4 files changed, 45 insertions(+), 14 deletions(-) create mode 100644 parm/jcb-gdas/algorithm/obstats/atmos/conventional_ps_template.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/obstats/aero/viirs_n20_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/aero/viirs_n20_template.yaml.j2 index 9994716ea..60bbbb2b2 100644 --- a/parm/jcb-gdas/algorithm/obstats/aero/viirs_n20_template.yaml.j2 +++ b/parm/jcb-gdas/algorithm/obstats/aero/viirs_n20_template.yaml.j2 @@ -12,10 +12,6 @@ statistics to compute: ['mean', 'count', 'RMS'] output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_aod.nc" domains to process: - - domain: - name: "Global" - first mask variable: latitude - first mask range: [-180,-100] - domain: name: "SH" first mask variable: latitude diff --git a/parm/jcb-gdas/algorithm/obstats/aero/viirs_npp_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/aero/viirs_npp_template.yaml.j2 index 3c4c5d7f7..60bbbb2b2 100644 --- a/parm/jcb-gdas/algorithm/obstats/aero/viirs_npp_template.yaml.j2 +++ b/parm/jcb-gdas/algorithm/obstats/aero/viirs_npp_template.yaml.j2 @@ -12,10 +12,6 @@ statistics to compute: ['mean', 'count', 'RMS'] output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_aod.nc" domains to process: - - domain: - name: "Global" - first mask variable: latitude - first mask range: [-180,-100] - domain: name: "SH" first mask variable: latitude @@ -47,4 +43,4 @@ first mask variable: latitude first mask range: [0,70] second mask variable: longitude - second mask range: [38, 180] \ No newline at end of file + second mask range: [38, 180] diff --git a/parm/jcb-gdas/algorithm/obstats/atmos/conventional_ps_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/atmos/conventional_ps_template.yaml.j2 new file mode 100644 index 000000000..b187ebf62 --- /dev/null +++ b/parm/jcb-gdas/algorithm/obstats/atmos/conventional_ps_template.yaml.j2 @@ -0,0 +1,43 @@ +- obs space: + name: sfc + obsdatain: + engine: + type: H5File + obsfile: {{ atmosphere_obsdatain_path }}/atmos/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc + simulated variables: ['stationPressure'] + observed variables: ['stationPressure'] + statistics to compute: ['mean', 'count', 'RMS'] + output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_atmos.nc" + domains to process: + - domain: + name: "SH" + first mask variable: latitude + first mask range: [-90,0] + - domain: + name: "NH" + first mask variable: latitude + first mask range: [0,90] + - domain: + name: "CONUS" + first mask variable: latitude + first mask range: [25,49] + second mask variable: longitude + second mask range: [-125,-66] + - domain: + name: "Europe" + first mask variable: latitude + first mask range: [35,70] + second mask variable: longitude + second mask range: [-11,38] + - domain: + name: "Africa" + first mask variable: latitude + first mask range: [-35,37] + second mask variable: longitude + second mask range: [-17,52] + - domain: + name: "Asia" + first mask variable: latitude + first mask range: [0,70] + second mask variable: longitude + second mask range: [38, 180] diff --git a/parm/jcb-gdas/algorithm/obstats/snow/ims_snow_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/snow/ims_snow_template.yaml.j2 index 7e9880392..059c5f348 100644 --- a/parm/jcb-gdas/algorithm/obstats/snow/ims_snow_template.yaml.j2 +++ b/parm/jcb-gdas/algorithm/obstats/snow/ims_snow_template.yaml.j2 @@ -9,10 +9,6 @@ statistics to compute: ['mean', 'count', 'RMS'] output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_snow.nc" domains to process: - - domain: - name: "Global" - first mask variable: latitude - first mask range: [-180,-100] - domain: name: "SH" first mask variable: latitude @@ -44,4 +40,4 @@ first mask variable: latitude first mask range: [0,70] second mask variable: longitude - second mask range: [38, 180] \ No newline at end of file + second mask range: [38, 180] From 768f68e065ed314525068a4d449e312b8999f425 Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Fri, 28 Mar 2025 17:39:16 +0000 Subject: [PATCH 097/199] group snow obs by stationID (#91) https://github.com/NOAA-EMC/GDASApp/issues/1576 discussion found reproducibility issues between PE counts for snow 2DVar. I tracked it down to the temporal thinning filter. When not explicitly grouping by stationID at IODA, reproducibility issues can occur. Thus, we should group by stationID when the obs space is created. --- parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 | 2 ++ parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 | 2 ++ parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 | 2 ++ 3 files changed, 6 insertions(+) diff --git a/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 index 1ad679260..0850057bb 100644 --- a/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 @@ -9,6 +9,8 @@ type: H5File obsfile: "{{snow_obsdatain_path}}/{{snow_obsdatain_prefix}}{{observation_from_jcb}}{{snow_obsdatain_suffix}}" missing file action: warn + obsgrouping: + group variables: [stationIdentification] obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 b/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 index b6151a7ad..ce3711075 100644 --- a/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 @@ -10,6 +10,8 @@ obsfile: "{{snow_obsdatain_path}}/{{snow_obsdatain_prefix}}sfcsno.tm00.bufr_d" mapping file: "{{snow_obsdatain_path}}/bufr_sfcsno_mapping.yaml" missing file action: warn + obsgrouping: + group variables: [stationIdentification] obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 index c38da080e..ebef03edf 100644 --- a/parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 @@ -9,6 +9,8 @@ type: H5File obsfile: "{{snow_obsdatain_path}}/{{snow_obsdatain_prefix}}{{observation_from_jcb}}.nc4" missing file action: warn + obsgrouping: + group variables: [stationIdentification] obsdataout: engine: type: H5File From b1605409ccb7f0aa09fb9099c1f12868a418db4c Mon Sep 17 00:00:00 2001 From: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com> Date: Mon, 31 Mar 2025 08:49:35 -0400 Subject: [PATCH 098/199] add state variables to inverse block to gsibec templates (#93) This PR adds the `state variables to inverse` block to - `model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2` - `model/atmosphere/atmosphere_background_error_static_gsibec.yaml.j2` Updates to saber at [dce89ba](https://github.com/JCSDA/saber/commit/dce89bab2223ffd00f98e25c28a5ee11bd67c542) and fv3-jedi at [7e8337d](https://github.com/JCSDA/fv3-jedi/commit/7e8337ddb7ebcbfcce3a62d5fb282ca092586e7d) require addition of a state variables to inverse block in GSIBEC yamls. See saber PR [#1022](https://github.com/JCSDA-internal/saber/pull/1022) and fv3-jedi PR [#1350](https://github.com/JCSDA-internal/fv3-jedi/pull/1350) for details. Resolves #92 The changes in this PR have been tested in GDASApp `feature/stable-nightly` and found to work. --- .../atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 | 3 +++ .../atmosphere_background_error_static_gsibec.yaml.j2 | 3 +++ 2 files changed, 6 insertions(+) diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 index 588091752..40e615ac0 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 @@ -19,6 +19,9 @@ components: processor layout x direction: {{atmosphere_layout_gsib_x}} processor layout y direction: {{atmosphere_layout_gsib_y}} debugging mode: false + state variables to inverse: [eastward_wind,northward_wind,air_temperature,air_pressure_at_surface, + water_vapor_mixing_ratio_wrt_moist_air,cloud_liquid_ice,cloud_liquid_water, + ozone_mass_mixing_ratio] linear variable change: linear variable change name: Control2Analysis input variables: [eastward_wind,northward_wind,air_temperature,air_pressure_at_surface, diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_gsibec.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_gsibec.yaml.j2 index 1133aaf7d..d77b49e49 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_gsibec.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_gsibec.yaml.j2 @@ -17,6 +17,9 @@ saber outer blocks: processor layout x direction: {{atmosphere_layout_gsib_x}} processor layout y direction: {{atmosphere_layout_gsib_y}} debugging mode: false + state variables to inverse: [eastward_wind,northward_wind,air_temperature,air_pressure_at_surface, + water_vapor_mixing_ratio_wrt_moist_air,cloud_liquid_ice,cloud_liquid_water, + ozone_mass_mixing_ratio] linear variable change: linear variable change name: Control2Analysis input variables: [eastward_wind,northward_wind,air_temperature,air_pressure_at_surface, From 708dea58a738107978ef4ff731b1300e1ab74789 Mon Sep 17 00:00:00 2001 From: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com> Date: Tue, 1 Apr 2025 12:32:05 -0400 Subject: [PATCH 099/199] Update atmospheric EnVar localization for use with GSIBEC (#97) --- .../atmosphere_3dvar_outer_loop_1.yaml.j2 | 1 + .../atmosphere_3dvar_outer_loop_2.yaml.j2 | 1 + ...ackground_error_hybrid_gsibec_bump.yaml.j2 | 260 +++++++++--------- 3 files changed, 131 insertions(+), 131 deletions(-) diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 index 3b65a686c..671fbd3e9 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 @@ -13,5 +13,6 @@ npy: {{atmosphere_npy_anl}} npz: {{atmosphere_npz_anl}} field metadata override: {{atmosphere_fv3jedi_files_path}}/fv3jedi_fieldmetadata_history.yaml + vert coordinate: logp diagnostics: departures: bkgmob diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 index 5ca7f8082..86344aecb 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 @@ -13,5 +13,6 @@ npy: {{atmosphere_npy_anl}} npz: {{atmosphere_npz_anl}} field metadata override: {{atmosphere_fv3jedi_files_path}}/fv3jedi_fieldmetadata_history.yaml + vert coordinate: logp diagnostics: departures: bkgmob1 diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 index 40e615ac0..80ac8ed98 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 @@ -53,7 +53,7 @@ components: ozone_mass_mixing_ratio ] read: general: - universe length-scale: 2500.0e3 + universe length-scale: 4641.0e3 drivers: multivariate strategy: duplicated compute nicas: true @@ -61,144 +61,142 @@ components: level for 2d variables: last nicas: resolution: 6 - min effective resolution: 0.2 # Note: this is very much not recommended (resolution too low), only - # used because covariance quality is not important for CI tests explicit length-scales: true horizontal length-scale: - groups: - common profile: - - 1300000.0 - - 1300000.0 - - 1300000.0 - - 1300000.0 - - 1300000.0 - - 1300000.0 - - 1300000.0 - - 1300000.0 - - 1300000.0 - - 1300000.0 - - 1300000.0 - - 1250000.0 - - 1250000.0 - - 1200000.0 - - 1200000.0 - - 1150000.0 - - 1150000.0 - - 1100000.0 - - 1100000.0 - - 1000000.0 - - 1000000.0 - - 1000000.0 - - 1000000.0 - - 1000000.0 - - 1000000.0 - - 1000000.0 - - 1000000.0 - - 1000000.0 - - 1000000.0 - - 1000000.0 - - 1000000.0 - - 1000000.0 - - 1000000.0 - - 1000000.0 - - 1000000.0 - - 1000000.0 - - 1000000.0 - - 1000000.0 - - 1000000.0 - - 950000.0 - - 950000.0 - - 900000.0 - - 900000.0 - - 850000.0 - - 850000.0 - - 800000.0 - - 800000.0 - - 750000.0 - - 750000.0 - - 665000.0 - - 665000.0 - - 585000.0 - - 585000.0 - - 510000.0 - - 510000.0 - - 440000.0 - - 440000.0 - - 390000.0 - - 390000.0 - - 380000.0 - - 380000.0 - - 370000.0 - - 370000.0 - - 360000.0 - - 360000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 - - 350000.0 + - 4641000.0 + - 4641000.0 + - 4641000.0 + - 4641000.0 + - 4641000.0 + - 4641000.0 + - 4641000.0 + - 4641000.0 + - 4641000.0 + - 4641000.0 + - 4641000.0 + - 4462500.0 + - 4462500.0 + - 4284000.0 + - 4284000.0 + - 4105500.0 + - 4105500.0 + - 3927000.0 + - 3927000.0 + - 3570000.0 + - 3570000.0 + - 3570000.0 + - 3570000.0 + - 3570000.0 + - 3570000.0 + - 3570000.0 + - 3570000.0 + - 3570000.0 + - 3570000.0 + - 3570000.0 + - 3570000.0 + - 3570000.0 + - 3570000.0 + - 3570000.0 + - 3570000.0 + - 3570000.0 + - 3570000.0 + - 3570000.0 + - 3570000.0 + - 3391500.0 + - 3391500.0 + - 3213000.0 + - 3213000.0 + - 3034500.0 + - 3034500.0 + - 2856000.0 + - 2856000.0 + - 2677500.0 + - 2677500.0 + - 2374050.0 + - 2374050.0 + - 2088450.0 + - 2088450.0 + - 1820700.0 + - 1820700.0 + - 1570800.0 + - 1570800.0 + - 1392300.0 + - 1392300.0 + - 1356600.0 + - 1356600.0 + - 1320900.0 + - 1320900.0 + - 1285200.0 + - 1285200.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 + - 1249500.0 vertical length-scale: - groups: - common - value: 0.3 + value: 1.8 linear variable change: linear variable change name: Control2Analysis input variables: [eastward_wind,northward_wind,air_temperature,air_pressure_at_surface, From 666df69edcd7ac8998e6867203ce20cee98e08fe Mon Sep 17 00:00:00 2001 From: Guillaume Vernieres Date: Tue, 1 Apr 2025 17:08:55 -0400 Subject: [PATCH 100/199] Added barrier island configs for B and obs (#82) - Fixes https://github.com/NOAA-EMC/GDASApp/issues/1506 - Fixes https://github.com/NOAA-EMC/GDASApp/issues/1514 --- .../marine/soca_setcorscales.yaml.j2 | 6 ++++ .../observations/marine/sst_generic.yaml.j2 | 31 ++++++++++++++----- 2 files changed, 30 insertions(+), 7 deletions(-) diff --git a/parm/jcb-gdas/algorithm/marine/soca_setcorscales.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_setcorscales.yaml.j2 index 9d07bef08..a30b68588 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_setcorscales.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_setcorscales.yaml.j2 @@ -7,6 +7,12 @@ date: "0001-01-01T00:00:00Z" corr variables: [sea_surface_height_above_geoid] +islands: + # Barrier Islands between the Pamlico Sound and the Atlantic + scale: 100.0 # in kilometers + lat: [ 34.636, 35.065, 35.49, 35.87, 36.28] + lon: [-76.52, -76.03, -75.46, -75.56, -75.78] + scales: vert layers: 5 # in units of layer sea_surface_height_above_geoid: diff --git a/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 index 30754a0b5..a567fc7df 100644 --- a/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 @@ -18,20 +18,21 @@ observation alias file: obsop_name_map.yaml obs error: covariance model: diagonal - obs filters: - - filter: Domain Check - where: - - variable: {name: GeoVaLs/sea_area_fraction} - minvalue: 0.9 + + obs pre filters: - filter: Bounds Check minvalue: -1.2 maxvalue: 41.0 - - filter: Background Check - absolute threshold: 5.0 - filter: Domain Check where: - variable: {name: ObsError/seaSurfaceTemperature} minvalue: 1.0e-8 + + obs prior filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + minvalue: 0.9 - filter: Domain Check where: - variable: { name: GeoVaLs/sea_ice_area_fraction} @@ -44,3 +45,19 @@ where: - variable: {name: GeoVaLs/distance_from_coast} minvalue: 50e3 + + obs post filters: + - filter: Perform Action # Pamlico Sound & Chesapeake Bay + action: + name: accept + where: + - variable: + name: MetaData/latitude + minvalue: 34.5 + maxvalue: 39.7 + - variable: + name: MetaData/longitude + minvalue: -76.88 + maxvalue: -74.5 + - filter: Background Check + absolute threshold: 5.0 From e66d25c131b10ac3dd767988ebbcf48bac88bb45 Mon Sep 17 00:00:00 2001 From: AndrewEichmann-NOAA <58948505+AndrewEichmann-NOAA@users.noreply.github.com> Date: Wed, 2 Apr 2025 13:32:09 -0400 Subject: [PATCH 101/199] Adds letkf localization block to marine observation config yamls (#94) The localization block is necessary to use the observation config yamls for LETKF. --- parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 | 9 +++++++++ .../jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 | 9 +++++++++ .../observations/marine/icec_amsr2_north.yaml.j2 | 9 +++++++++ .../observations/marine/icec_amsr2_south.yaml.j2 | 9 +++++++++ .../jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 | 9 +++++++++ .../observations/marine/icec_atms_n20_l2.yaml.j2 | 9 +++++++++ .../observations/marine/icec_atms_n21_l2.yaml.j2 | 9 +++++++++ .../observations/marine/icec_atms_npp_l2.yaml.j2 | 9 +++++++++ .../jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 | 9 +++++++++ .../observations/marine/icec_ssmis_f17_l2.yaml.j2 | 9 +++++++++ .../observations/marine/icec_viirs_n20_l2_north.yaml.j2 | 9 +++++++++ .../observations/marine/icec_viirs_n20_l2_south.yaml.j2 | 9 +++++++++ .../observations/marine/insitu_profile_argo.yaml.j2 | 9 +++++++++ .../observations/marine/insitu_profile_bathy.yaml.j2 | 9 +++++++++ .../observations/marine/insitu_profile_dbuoy.yaml.j2 | 9 +++++++++ .../observations/marine/insitu_profile_dbuoyb.yaml.j2 | 9 +++++++++ .../observations/marine/insitu_profile_glider.yaml.j2 | 9 +++++++++ .../marine/insitu_profile_marinemammal.yaml.j2 | 9 +++++++++ .../observations/marine/insitu_profile_mbuoy.yaml.j2 | 9 +++++++++ .../observations/marine/insitu_profile_mbuoyb.yaml.j2 | 9 +++++++++ .../observations/marine/insitu_profile_tesac.yaml.j2 | 9 +++++++++ .../marine/insitu_profile_tesac_salinity.yaml.j2 | 9 +++++++++ .../observations/marine/insitu_profile_tropical.yaml.j2 | 9 +++++++++ .../observations/marine/insitu_profile_xbtctd.yaml.j2 | 9 +++++++++ .../observations/marine/insitu_salt_profile_argo.yaml.j2 | 9 +++++++++ .../observations/marine/insitu_surface_altkob.yaml.j2 | 9 +++++++++ .../observations/marine/insitu_surface_drifter.yaml.j2 | 9 +++++++++ .../observations/marine/insitu_surface_trkob.yaml.j2 | 9 +++++++++ .../marine/insitu_surface_trkob_salinity.yaml.j2 | 9 +++++++++ .../observations/marine/insitu_temp_profile_argo.yaml.j2 | 9 +++++++++ parm/jcb-gdas/observations/marine/sss_smap_l2.yaml.j2 | 9 +++++++++ parm/jcb-gdas/observations/marine/sss_smos_l2.yaml.j2 | 9 +++++++++ parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 | 9 +++++++++ 33 files changed, 297 insertions(+) diff --git a/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 b/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 index 1fc2f054e..c7e3a1004 100644 --- a/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 @@ -60,3 +60,12 @@ name: MetaData/longitude minvalue: 32 maxvalue: 56 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 index 33eeb2382..5fd44160e 100644 --- a/parm/jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 @@ -55,3 +55,12 @@ where: - variable: {name: GeoVaLs/sea_surface_temperature} maxvalue: -4.0 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 index 6d88cc36f..e1a92201d 100644 --- a/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 @@ -49,3 +49,12 @@ # select_median: true # action: # name: reduce obs space +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 index 6d88cc36f..e1a92201d 100644 --- a/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 @@ -49,3 +49,12 @@ # select_median: true # action: # name: reduce obs space +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 index 33eeb2382..5fd44160e 100644 --- a/parm/jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 @@ -55,3 +55,12 @@ where: - variable: {name: GeoVaLs/sea_surface_temperature} maxvalue: -4.0 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/icec_atms_n20_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_atms_n20_l2.yaml.j2 index 33eeb2382..5fd44160e 100644 --- a/parm/jcb-gdas/observations/marine/icec_atms_n20_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_atms_n20_l2.yaml.j2 @@ -55,3 +55,12 @@ where: - variable: {name: GeoVaLs/sea_surface_temperature} maxvalue: -4.0 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/icec_atms_n21_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_atms_n21_l2.yaml.j2 index 33eeb2382..5fd44160e 100644 --- a/parm/jcb-gdas/observations/marine/icec_atms_n21_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_atms_n21_l2.yaml.j2 @@ -55,3 +55,12 @@ where: - variable: {name: GeoVaLs/sea_surface_temperature} maxvalue: -4.0 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/icec_atms_npp_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_atms_npp_l2.yaml.j2 index 33eeb2382..5fd44160e 100644 --- a/parm/jcb-gdas/observations/marine/icec_atms_npp_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_atms_npp_l2.yaml.j2 @@ -55,3 +55,12 @@ where: - variable: {name: GeoVaLs/sea_surface_temperature} maxvalue: -4.0 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 index 4018e17f2..946c2d42a 100644 --- a/parm/jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 @@ -49,3 +49,12 @@ where: - variable: {name: GeoVaLs/sea_surface_temperature} maxvalue: -4.0 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 index 33eeb2382..5fd44160e 100644 --- a/parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 @@ -55,3 +55,12 @@ where: - variable: {name: GeoVaLs/sea_surface_temperature} maxvalue: -4.0 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_north.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_north.yaml.j2 index 33eeb2382..5fd44160e 100644 --- a/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_north.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_north.yaml.j2 @@ -55,3 +55,12 @@ where: - variable: {name: GeoVaLs/sea_surface_temperature} maxvalue: -4.0 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_south.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_south.yaml.j2 index 33eeb2382..5fd44160e 100644 --- a/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_south.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_south.yaml.j2 @@ -55,3 +55,12 @@ where: - variable: {name: GeoVaLs/sea_surface_temperature} maxvalue: -4.0 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_argo.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_argo.yaml.j2 index 12b09fa5d..f26cd42f0 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_argo.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_argo.yaml.j2 @@ -336,3 +336,12 @@ - ObsError/salinity coefs: - 1000.0 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 index e379c3b35..1811610c7 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 @@ -26,3 +26,12 @@ - ObsError/waterTemperature coefs: - 1000.0 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_dbuoy.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_dbuoy.yaml.j2 index e8a9ca3bc..2848b56aa 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_dbuoy.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_dbuoy.yaml.j2 @@ -32,3 +32,12 @@ coefs: - 1000.0 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_dbuoyb.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_dbuoyb.yaml.j2 index 8cdc4e166..65a5a704e 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_dbuoyb.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_dbuoyb.yaml.j2 @@ -31,3 +31,12 @@ - ObsError/waterTemperature coefs: - 1000.0 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_glider.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_glider.yaml.j2 index 8cdc4e166..65a5a704e 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_glider.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_glider.yaml.j2 @@ -31,3 +31,12 @@ - ObsError/waterTemperature coefs: - 1000.0 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_marinemammal.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_marinemammal.yaml.j2 index 8cdc4e166..65a5a704e 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_marinemammal.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_marinemammal.yaml.j2 @@ -31,3 +31,12 @@ - ObsError/waterTemperature coefs: - 1000.0 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_mbuoy.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_mbuoy.yaml.j2 index 8cdc4e166..65a5a704e 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_mbuoy.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_mbuoy.yaml.j2 @@ -31,3 +31,12 @@ - ObsError/waterTemperature coefs: - 1000.0 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_mbuoyb.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_mbuoyb.yaml.j2 index 8cdc4e166..65a5a704e 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_mbuoyb.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_mbuoyb.yaml.j2 @@ -31,3 +31,12 @@ - ObsError/waterTemperature coefs: - 1000.0 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_tesac.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_tesac.yaml.j2 index d0986b9a4..cc6e98786 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_tesac.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_tesac.yaml.j2 @@ -29,3 +29,12 @@ - ObsError/waterTemperature coefs: - 1000.0 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_tesac_salinity.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_tesac_salinity.yaml.j2 index 4e8b3bc17..1fdc5d262 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_tesac_salinity.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_tesac_salinity.yaml.j2 @@ -31,3 +31,12 @@ - ObsError/salinity coefs: - 1000.0 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_tropical.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_tropical.yaml.j2 index 8cdc4e166..65a5a704e 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_tropical.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_tropical.yaml.j2 @@ -31,3 +31,12 @@ - ObsError/waterTemperature coefs: - 1000.0 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_xbtctd.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_xbtctd.yaml.j2 index 8cdc4e166..65a5a704e 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_xbtctd.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_xbtctd.yaml.j2 @@ -31,3 +31,12 @@ - ObsError/waterTemperature coefs: - 1000.0 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 index 8db7302fb..0448247ce 100644 --- a/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 @@ -176,3 +176,12 @@ min grid mult: 2.0 min value: 200.0e3 max value: 900.0e3 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/insitu_surface_altkob.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_surface_altkob.yaml.j2 index cf7da6427..fbe5e1b77 100644 --- a/parm/jcb-gdas/observations/marine/insitu_surface_altkob.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_surface_altkob.yaml.j2 @@ -27,3 +27,12 @@ - ObsError/seaSurfaceTemperature coefs: - 1000.0 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/insitu_surface_drifter.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_surface_drifter.yaml.j2 index 8cdc4e166..65a5a704e 100644 --- a/parm/jcb-gdas/observations/marine/insitu_surface_drifter.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_surface_drifter.yaml.j2 @@ -31,3 +31,12 @@ - ObsError/waterTemperature coefs: - 1000.0 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/insitu_surface_trkob.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_surface_trkob.yaml.j2 index 833fdac9c..7a20a4217 100644 --- a/parm/jcb-gdas/observations/marine/insitu_surface_trkob.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_surface_trkob.yaml.j2 @@ -27,3 +27,12 @@ - ObsError/seaSurfaceTemperature coefs: - 1000.0 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/insitu_surface_trkob_salinity.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_surface_trkob_salinity.yaml.j2 index 3bf2085ff..c7d05e8c2 100644 --- a/parm/jcb-gdas/observations/marine/insitu_surface_trkob_salinity.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_surface_trkob_salinity.yaml.j2 @@ -27,3 +27,12 @@ - ObsError/seaSurfaceSalinity coefs: - 1000.0 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 index 98fac26ee..ff640d832 100644 --- a/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 @@ -171,3 +171,12 @@ min grid mult: 2.0 min value: 200.0e3 max value: 900.0e3 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/sss_smap_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/sss_smap_l2.yaml.j2 index 20b645b2f..6dd8d64cf 100644 --- a/parm/jcb-gdas/observations/marine/sss_smap_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sss_smap_l2.yaml.j2 @@ -42,3 +42,12 @@ where: - variable: {name: GeoVaLs/distance_from_coast} minvalue: 100e3 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/sss_smos_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/sss_smos_l2.yaml.j2 index 20b645b2f..6dd8d64cf 100644 --- a/parm/jcb-gdas/observations/marine/sss_smos_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sss_smos_l2.yaml.j2 @@ -42,3 +42,12 @@ where: - variable: {name: GeoVaLs/distance_from_coast} minvalue: 100e3 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 index a567fc7df..31b5d2d05 100644 --- a/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 @@ -45,6 +45,15 @@ where: - variable: {name: GeoVaLs/distance_from_coast} minvalue: 50e3 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} obs post filters: - filter: Perform Action # Pamlico Sound & Chesapeake Bay From aa5c43f98c7c6a1244f31193bd28e744a53158c0 Mon Sep 17 00:00:00 2001 From: jianjunj Date: Wed, 2 Apr 2025 15:40:06 -0400 Subject: [PATCH 102/199] Add precipitable clouds in microwave all-sky assimilations (updated) (#74) This is an update version of the PR https://github.com/NOAA-EMC/jcb-gdas/pull/63 which resolves the issue https://github.com/NOAA-EMC/GDASApp/issues/1333 . The main update is that calculations of cloud fraction and cloud particle radius by the Thompson microphysics scheme are added. Here is the summary of other added configurations in the original PR #63: Add rain, snow, and graupel clouds for CRTM inputs. Add a prior filter to zero clouds over non-water surfaces. Add an option of clouds in the Hydrometeo check. Add gross error check threshold for precipition impacted channels. Add cold-air-outbreak filters. Read cloud fraction from GeoVals. Render list of not-bias corrected channels from JCB tables. Add configurations for atms_n21. It depends on two UFO PRs https://github.com/JCSDA-internal/ufo/pull/3600 https://github.com/JCSDA-internal/ufo/pull/3573 It needs **the develop branch in "global-workflow/sorc/gdas.cd/sorc/jcb"** in order to rend non-bias corrected channels. These UFO PRs are being revised. I'll inform you when it is ready to be merged. --------- Co-authored-by: Cory Martin --- .../atmosphere/atms_n21.yaml | 43 ++ .../observations/atmosphere/amsua_n19.yaml.j2 | 137 +++- .../observations/atmosphere/atms_n20.yaml.j2 | 294 +++++--- .../observations/atmosphere/atms_n21.yaml.j2 | 625 ++++++++++++++++++ .../observations/atmosphere/atms_npp.yaml.j2 | 280 +++++--- 5 files changed, 1166 insertions(+), 213 deletions(-) create mode 100644 parm/jcb-gdas/observation_chronicle/atmosphere/atms_n21.yaml create mode 100644 parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/atms_n21.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/atms_n21.yaml new file mode 100644 index 000000000..3311e650a --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/atmosphere/atms_n21.yaml @@ -0,0 +1,43 @@ +# Instrument metadata +# ------------------- +commissioned: 2023-05-01T12:00:00 + +observer_type: satellite # Type of chronicle to use + +# Instrument initial configuration +# -------------------------------- +channel_variables: + simulated: min + active: min + biascorrtd: min + error: max + error_cld: max + ermax: max + var_b: min + var_pg: min + icld_det: min + icloud: min + iaerosol: min +channel_values: + 1: [ 1, 1, 1, 4.50, 20.00, 4.50, 10.00, 0.00, -2, 1, -1 ] + 2: [ 1, 1, 1, 4.50, 25.00, 4.50, 10.00, 0.00, -2, 1, -1 ] + 3: [ 1, 1, 1, 4.50, 12.00, 3.00, 10.00, 0.00, -2, 1, -1 ] + 4: [ 1, 1, 1, 2.50, 7.00, 3.00, 10.00, 0.00, -2, 1, -1 ] + 5: [ 1, 1, 1, 0.55, 3.50, 1.00, 10.00, 0.00, -2, 1, -1 ] + 6: [ 1, 1, 1, 0.30, 3.00, 1.00, 10.00, 0.00, -2, 1, -1 ] + 7: [ 1, 1, 1, 0.30, 0.80, 1.00, 10.00, 0.00, -2, 1, -1 ] + 8: [ 1, 1, 1, 0.40, 0.40, 1.00, 10.00, 0.00, -2, 1, -1 ] + 9: [ 1, 1, 1, 0.40, 0.40, 1.00, 10.00, 0.00, -2, 1, -1 ] + 10: [ 1, 1, 1, 0.40, 0.40, 1.00, 10.00, 0.00, -2, 1, -1 ] + 11: [ 1, 1, 1, 0.45, 0.45, 1.00, 10.00, 0.00, -2, 1, -1 ] + 12: [ 1, 1, 1, 0.45, 0.45, 1.00, 10.00, 0.00, -2, 1, -1 ] + 13: [ 1, 1, 1, 0.55, 0.55, 1.00, 10.00, 0.00, -2, 1, -1 ] + 14: [ 1, 1, 1, 0.80, 0.80, 2.00, 10.00, 0.00, -2, 1, -1 ] + 15: [ 1, 1, 0, 4.00, 4.00, 4.50, 10.00, 0.00, -2, 1, -1 ] + 16: [ 1, 1, 1, 4.00, 19.00, 4.50, 10.00, 0.00, -2, 1, -1 ] + 17: [ 1, 1, 1, 4.00, 30.00, 2.00, 10.00, 0.00, -2, 1, -1 ] + 18: [ 1, 1, 1, 3.50, 25.00, 2.00, 10.00, 0.00, -2, 1, -1 ] + 19: [ 1, 1, 1, 3.00, 16.50, 2.00, 10.00, 0.00, -2, 1, -1 ] + 20: [ 1, 1, 1, 3.00, 12.00, 2.00, 10.00, 0.00, -2, 1, -1 ] + 21: [ 1, 1, 1, 3.00, 9.00, 2.00, 10.00, 0.00, -2, 1, -1 ] + 22: [ 1, 1, 1, 3.00, 6.50, 2.00, 10.00, 0.00, -2, 1, -1 ] diff --git a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 index dba4162db..c1f624250 100644 --- a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 @@ -21,9 +21,13 @@ # -------------------- obs operator: name: CRTM - Absorbers: [H2O, O3] - Clouds: [Water, Ice] - Cloud_Fraction: 1.0 + Absorbers: [H2O, O3, CO2] + Clouds: &{{observation_from_jcb}}_clouds [Water, Ice, Rain, Snow, Graupel] +# Cloud_Fraction: 1.0 +# ---methods for cloud fraction and radii in fov: thompson, or none + method for cloud fraction within fov: thompson + method for hydrometeor effective radii within fov: thompson + Cloud_Seeding: true obs options: Sensor_ID: &{{observation_from_jcb}}_sensor_id amsua_n19 EndianType: little_endian @@ -34,6 +38,8 @@ obs bias: input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" + variables without bc: [brightnessTemperature] + channels: {{ get_satellite_variable(observation_from_jcb, "not_biascorrtd") }} variational bc: predictors: - name: constant @@ -64,7 +70,25 @@ # Observation Filters (QC) # ------------------------ - obs filters: + obs prior filters: + # Zero Atmospheric clouds in CRTM where water_area_fraction < 0.99 + - filter: Variable Assignment + assignments: + - name: MetaData/zeroCloudInCRTM + type: int + function: + name: ObsFunction/Conditional + options: +# firstmatchingcase: false + defaultvalue: 0 # Will not zero clouds by default + cases: + - where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + value: 1 # Will zero clouds by default + + obs post filters: - filter: BlackList filter variables: - name: brightnessTemperature @@ -124,6 +148,7 @@ channels: *{{observation_from_jcb}}_simulated_channels options: channels: *{{observation_from_jcb}}_simulated_channels + Clouds: *{{observation_from_jcb}}_clouds obserr_clearsky: *{{observation_from_jcb}}_err0 clwret_function: name: ObsFunction/CLWRetMW @@ -301,6 +326,8 @@ x1: *{{observation_from_jcb}}_x1 err0: *{{observation_from_jcb}}_err0 err1: *{{observation_from_jcb}}_err1 + threshold: 3 + threshold_precip: 2.5 obserr_bound_max: [{{ get_satellite_variable(observation_from_jcb, "ermax") }}] action: name: reject @@ -334,8 +361,108 @@ minvalue: 1.0e-12 action: name: reject +# Combined ColumnLiquidCloud + ColumnIceCloud + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnLiquidIceCloud + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: GeoVaLs/mass_content_of_cloud_liquid_water_in_atmosphere_column + - name: GeoVaLs/mass_content_of_cloud_ice_in_atmosphere_column + coefs: [1, 1] +# replace zero by 9.99e-7 in DerivedMetaData/ColumnLiquidIceCloud + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnLiquidIceCloud + type: float + value: 9.99e-07 + where: + - variable: + name: DerivedMetaData/ColumnLiquidIceCloud + maxvalue: 9.99e-07 - +# ratio of liquid /(Liquid+Ice) + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnRatioLiquidCloud + type: float + function: + name: ObsFunction/ElementMultiply + options: + variables: + - name: GeoVaLs/mass_content_of_cloud_liquid_water_in_atmosphere_column + - name: DerivedMetaData/ColumnLiquidIceCloud + exponents: [1,-1] + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnRatioLiquidCloud + type: float + value: 0.0 + where: + - variable: + name: DerivedMetaData/ColumnLiquidIceCloud + maxvalue: 1.0e-06 +# TotalColumnVaporGuess + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/TotalColumnVaporGuess + type: float + function: + name: ObsFunction/TotalColumnVaporGuess +# potentialTemperature at surface + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureSurface + type: float + function: + name: PotentialTemperatureFromTemperature@ObsFunction + options: + use surface pressure: true +# potentialTemperature near 700 hPa + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureAt700hPa + type: float + function: + name: PotentialTemperatureFromTemperature@ObsFunction + options: + use surface pressure: false + pressure to evaluate potential temperature: 70000.0 +# stability = Difference between PotentialTemperatur values + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureDiff + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: DerivedMetaData/PotentialTemperatureAt700hPa + - name: DerivedMetaData/PotentialTemperatureSurface + coefs: [1, -1] +# cold-air-outbreak (cao) check for all-sky DA with Rain, Snow, or Graupel + - filter: BlackList + filter variables: + - name: brightnessTemperature + channels: 1-6, 15 + where: + - variable: + name: ObsFunction/TotalColumnVaporGuess + maxvalue: 8.0 + - variable: + name: DerivedMetaData/ColumnRatioLiquidCloud + maxvalue: 0.5 + - variable: + name: DerivedMetaData/PotentialTemperatureDiff + maxvalue: 12.0 + - variable: + name: GeoVaLs/water_area_fraction + minvalue: 0.99 + action: + name: reject # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: diff --git a/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 index c16a22c51..a46ef0883 100644 --- a/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 @@ -22,8 +22,11 @@ obs operator: name: CRTM Absorbers: [H2O, O3, CO2] - Clouds: [Water, Ice] - Cloud_Fraction: 1.0 + Clouds: &{{observation_from_jcb}}_clouds [Water, Ice, Rain, Snow, Graupel] +# Cloud_Fraction: 1.0 +# ---methods for cloud fraction and radii in fov: thompson, or none + method for cloud fraction within fov: thompson + method for hydrometeor effective radii within fov: thompson Cloud_Seeding: true obs options: Sensor_ID: &{{observation_from_jcb}}_sensor_id atms_n20 @@ -37,6 +40,8 @@ obs bias: input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" + variables without bc: [brightnessTemperature] + channels: {{ get_satellite_variable(observation_from_jcb, "not_biascorrtd") }} variational bc: predictors: - name: constant @@ -65,39 +70,48 @@ ratio for small dataset: 2.0 output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" - # Observation Pre Filters (QC) - # ---------------------------- + # ------------------------ obs pre filters: - # Step 0-A: Create Diagnostic Flags - - filter: Create Diagnostic Flags + # Remove Observations from the Edge of the Scan + - filter: Domain Check filter variables: - name: brightnessTemperature - channels: *{{observation_from_jcb}}_simulated_channels - flags: - - name: ScanEdgeRemoval - initial value: false - force reinitialization: false - - name: Thinning - initial value: false - force reinitialization: false - - name: CLWRetrievalCheck - initial value: false - force reinitialization: false - - name: WindowChannelExtremeResidual - initial value: false - force reinitialization: false - - name: HydrometeorCheck - initial value: false - force reinitialization: false - - name: GrossCheck - initial value: false - force reinitialization: false - - name: InterChannelConsistency - initial value: false - force reinitialization: false - - name: UseflagCheck - initial value: false - force reinitialization: false + channels: 1-22 + where: + - variable: + name: MetaData/sensorScanPosition + is_in: 7-90 + action: + name: reduce obs space + + # Data Thinning + - filter: Gaussian Thinning + horizontal_mesh: 145 + use_reduced_horizontal_grid: true + distance_norm: geodesic + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + action: + name: reduce obs space + + # ------------------------ + obs prior filters: + # Zero Atmospheric clouds in CRTM where water_area_fraction < 0.99 + - filter: Variable Assignment + assignments: + - name: MetaData/zeroCloudInCRTM + type: int + function: + name: ObsFunction/Conditional + options: +# firstmatchingcase: false + defaultvalue: 0 # Will not zero clouds by default + cases: + - where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + value: 1 # Will zero clouds by default # Observation Post Filters (QC) # ----------------------------- @@ -114,6 +128,7 @@ clwret_ch238: 1 clwret_ch314: 2 clwret_types: [ObsValue] + # Calculate CLW retrieved from observation - filter: Variable Assignment assignments: @@ -125,6 +140,7 @@ clwret_ch238: 1 clwret_ch314: 2 clwret_types: [HofX] + # Calculate symmetric retrieved CLW - filter: Variable Assignment assignments: @@ -166,6 +182,7 @@ scatret_ch314: 2 scatret_ch890: 16 scatret_types: [ObsValue] + # Calculate CLW obs/bkg match index - filter: Variable Assignment assignments: @@ -184,6 +201,7 @@ clwret_clearsky: [0.030, 0.030, 0.030, 0.020, 0.030, 0.080, 0.150, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.020, 0.030, 0.030, 0.030, 0.030, 0.050, 0.100] + # Calculate symmetric observation error - filter: Variable Assignment assignments: @@ -209,6 +227,7 @@ err1: [20.000, 25.000, 12.000, 7.000, 3.500, 3.000, 0.800, 0.400, 0.400, 0.400, 0.450, 0.450, 0.550, 0.800, 4.000, 19.000, 30.000, 25.000, 16.500, 12.000, 9.000, 6.500] + # Calculate Innovation@DerivedMetaData - filter: Variable Assignment assignments: @@ -225,6 +244,7 @@ - name: brightnessTemperature@HofX channels: *{{observation_from_jcb}}_simulated_channels coefs: [1, -1] + # Step 0-C: Assign Initial All-Sky Observation Error - filter: Perform Action filter variables: @@ -235,33 +255,8 @@ error function: name: InitialObsError@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels - # Step 1: Remove Observations from the Edge of the Scan - - filter: Domain Check - filter variables: - - name: brightnessTemperature - channels: 1-22 - where: - - variable: - name: MetaData/sensorScanPosition - is_in: 7-90 - actions: - - name: set - flag: ScanEdgeRemoval - - name: reject - - # Step 2: Data Thinning - - filter: Gaussian Thinning - horizontal_mesh: 145 - use_reduced_horizontal_grid: true - distance_norm: geodesic - # round_horizontal_bin_count_to_nearest: true - # partition_longitude_bins_using_mesh: true - actions: - - name: set - flag: Thinning - - name: reject - # Step 3A: CLW Retrieval Check (observation_based) + # Step 1A: CLW Retrieval Check (observation_based) - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -269,12 +264,10 @@ test variables: - name: CLWRetFromObs@DerivedMetaData maxvalue: 999.0 - actions: - - name: set - flag: CLWRetrievalCheck - - name: reject + action: + name: reject - # Step 3B: CLW Retrieval Check (background_based) + # Step 1B: CLW Retrieval Check (background_based) - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -282,12 +275,10 @@ test variables: - name: CLWRetFromBkg@DerivedMetaData maxvalue: 999.0 - actions: - - name: set - flag: CLWRetrievalCheck - - name: reject + action: + name: reject - # Step 4: Window Channel Sanity Check + # Step 2: Window Channel Sanity Check - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -298,12 +289,10 @@ maxvalue: 200.0 minvalue: -200.0 flag all filter variables if any test variable is out of bounds: true - actions: - - name: set - flag: WindowChannelExtremeResidual - - name: reject + action: + name: reject - # Step 5: Hydrometeor Check (cloud/precipitation affected chanels) + # Step 3: Hydrometeor Check (cloud/precipitation affected chanels) - filter: Variable Assignment assignments: - name: DerivedMetaData/HydrometeorCheckATMS @@ -322,6 +311,7 @@ obserr_function: name: InitialObsError@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels + Clouds: *{{observation_from_jcb}}_clouds - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -330,13 +320,10 @@ - name: DerivedMetaData/HydrometeorCheckATMS channels: *{{observation_from_jcb}}_simulated_channels maxvalue: 0.0 - actions: - - name: set - flag: HydrometeorCheck - ignore: rejected observations - - name: reject + action: + name: reject - # Step 6: Observation Error Inflation based on Topography Check + # Step 4: Observation Error Inflation based on Topography Check - filter: Variable Assignment assignments: - name: ObsErrorFactorTopo@DerivedMetaData @@ -348,6 +335,7 @@ options: sensor: *{{observation_from_jcb}}_sensor_id channels: *{{observation_from_jcb}}_simulated_channels + - filter: Perform Action filter variables: - name: brightnessTemperature @@ -357,7 +345,8 @@ inflation variable: name: ObsErrorFactorTopo@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels - # Step 7: Obs Error Inflation based on TOA Transmittancec Check + + # Step 5: Obs Error Inflation based on TOA Transmittancec Check - filter: Variable Assignment assignments: - name: ObsErrorFactorTransmitTop@DerivedMetaData @@ -368,6 +357,7 @@ channels: *{{observation_from_jcb}}_simulated_channels options: channels: *{{observation_from_jcb}}_simulated_channels + - filter: Perform Action filter variables: - name: brightnessTemperature @@ -377,7 +367,8 @@ inflation variable: name: ObsErrorFactorTransmitTop@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels - # Step 8: Observation Error Inflation based on Surface Jacobian Check + + # Step 6: Observation Error Inflation based on Surface Jacobian Check - filter: Variable Assignment assignments: - name: ObsErrorFactorSurfJacobian@DerivedMetaData @@ -391,6 +382,7 @@ channels: *{{observation_from_jcb}}_simulated_channels obserr_demisf: [0.010, 0.020, 0.015, 0.020, 0.200] obserr_dtempf: [0.500, 2.000, 1.000, 2.000, 4.500] + - filter: Perform Action filter variables: - name: brightnessTemperature @@ -400,7 +392,8 @@ inflation variable: name: ObsErrorFactorSurfJacobian@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels - # Step 9: Situation Dependent Check + + # Step 7: Situation Dependent Check - filter: Variable Assignment assignments: - name: ObsErrorFactorSituDepend@DerivedMetaData @@ -427,6 +420,7 @@ obserr_clearsky: [4.500, 4.500, 4.500, 2.500, 0.550, 0.300, 0.300, 0.400, 0.400, 0.400, 0.450, 0.450, 0.550, 0.800, 4.000, 4.000, 4.000, 3.500, 3.000, 3.000, 3.000, 3.000] + - filter: Perform Action filter variables: - name: brightnessTemperature @@ -436,8 +430,9 @@ inflation variable: name: ObsErrorFactorSituDepend@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels - # Step 10: Gross check - # Remove data if abs(Obs-HofX) > absolute threhold + + # Step 8: Gross check + # Remove data if abs(Obs-HofX) > absolute threshold - filter: Variable Assignment assignments: - name: ObsErrorFactorLat@DerivedMetaData @@ -446,6 +441,7 @@ name: ObsErrorFactorLatRad@ObsFunction options: latitude_parameters: [25.0, 0.25, 0.04, 3.0] + - filter: Variable Assignment assignments: - name: ObsErrorBound@DerivedMetaData @@ -470,9 +466,11 @@ obserr_function: name: InitialObsError@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels - threhold: 3 + threshold: 3 + threshold_precip: 2.5 obserr_bound_max: [4.5, 4.5, 3.0, 3.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 2.0, 4.5, 4.5, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0] + - filter: Background Check filter variables: - name: brightnessTemperature @@ -480,13 +478,10 @@ function absolute threshold: - name: ObsErrorBound@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels - actions: - - name: set - flag: GrossCheck - ignore: rejected observations - - name: reject + action: + name: reject - # Step 11: Inter-Channel Check + # Step 9: Inter-Channel Check - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -501,13 +496,10 @@ use_flag: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 4, 1, 1, 1, 1, 1, 1, 1] maxvalue: 1.0e-12 - actions: - - name: set - flag: InterChannelConsistency - ignore: rejected observations - - name: reject + action: + name: reject - # Step 12: Useflag Check + # Step 10: Useflag Check - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -520,13 +512,109 @@ use_flag: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 4, 1, 1, 1, 1, 1, 1, 1] minvalue: 1.0e-12 - actions: - - name: set - flag: UseflagCheck - ignore: rejected observations - - name: reject - + action: + name: reject + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnLiquidIceCloud + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: GeoVaLs/mass_content_of_cloud_liquid_water_in_atmosphere_column + - name: GeoVaLs/mass_content_of_cloud_ice_in_atmosphere_column + coefs: [1, 1] +# replace zero by 9.99e-7 in DerivedMetaData/ColumnLiquidIceCloud + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnLiquidIceCloud + type: float + value: 9.99e-07 + where: + - variable: + name: DerivedMetaData/ColumnLiquidIceCloud + maxvalue: 9.99e-07 +# ratio of liquid /(Liquid+Ice) + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnRatioLiquidCloud + type: float + function: + name: ObsFunction/ElementMultiply + options: + variables: + - name: GeoVaLs/mass_content_of_cloud_liquid_water_in_atmosphere_column + - name: DerivedMetaData/ColumnLiquidIceCloud + exponents: [1,-1] + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnRatioLiquidCloud + type: float + value: 0.0 + where: + - variable: + name: DerivedMetaData/ColumnLiquidIceCloud + maxvalue: 1.0e-06 +# TotalColumnVaporGuess + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/TotalColumnVaporGuess + type: float + function: + name: ObsFunction/TotalColumnVaporGuess +# potentialTemperature at surface + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureSurface + type: float + function: + name: PotentialTemperatureFromTemperature@ObsFunction + options: + use surface pressure: true +# potentialTemperature near 700 hPa + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureAt700hPa + type: float + function: + name: PotentialTemperatureFromTemperature@ObsFunction + options: + use surface pressure: false + pressure to evaluate potential temperature: 70000.0 +# stability = Difference between PotentialTemperatur values + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureDiff + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: DerivedMetaData/PotentialTemperatureAt700hPa + - name: DerivedMetaData/PotentialTemperatureSurface + coefs: [1, -1] +# cold-air-outbreak (cao) check for all-sky DA with Rain, Snow, or Graupel + - filter: BlackList + filter variables: + - name: brightnessTemperature + channels: 1-7, 16-22 + where: + - variable: + name: ObsFunction/TotalColumnVaporGuess + maxvalue: 8.0 + - variable: + name: DerivedMetaData/ColumnRatioLiquidCloud + maxvalue: 0.5 + - variable: + name: DerivedMetaData/PotentialTemperatureDiff + maxvalue: 12.0 + - variable: + name: GeoVaLs/water_area_fraction + minvalue: 0.99 + action: + name: reject # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: diff --git a/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 new file mode 100644 index 000000000..a631473a3 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 @@ -0,0 +1,625 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: ATMS N21 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [brightnessTemperature] + channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} + + # Observation Operator + # -------------------- + obs operator: + name: CRTM + Absorbers: [H2O, O3, CO2] + Clouds: &{{observation_from_jcb}}_clouds [Water, Ice, Rain, Snow, Graupel] +# Cloud_Fraction: 1.0 +# ---methods for cloud fraction and radii in fov: thompson, or none + method for cloud fraction within fov: thompson + method for hydrometeor effective radii within fov: thompson + Cloud_Seeding: true + obs options: + Sensor_ID: &{{observation_from_jcb}}_sensor_id atms_n21 + EndianType: little_endian + CoefficientPath: "{{crtm_coefficient_path}}" + linear obs operator: + Absorbers: [H2O, O3] + + # Observation Bias Correction (VarBC) + # ----------------------------------- + obs bias: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" + variables without bc: [brightnessTemperature] + channels: {{ get_satellite_variable(observation_from_jcb, "not_biascorrtd") }} + variational bc: + predictors: + - name: constant + - name: lapseRate + order: 2 + tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" + - name: lapseRate + tlapse: *{{observation_from_jcb}}_tlapse + - name: emissivityJacobian + - name: sensorScanAngle + order: 4 + - name: sensorScanAngle + order: 3 + - name: sensorScanAngle + order: 2 + - name: sensorScanAngle + covariance: + minimal required obs number: 20 + variance range: [1.0e-6, 10.0] + step size: 1.0e-4 + largest analysis variance: 10000.0 + prior: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiascovin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovin_suffix}}" + inflation: + ratio: 1.1 + ratio for small dataset: 2.0 + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" + + # ------------------------ + obs pre filters: + # Remove Observations from the Edge of the Scan + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: 1-22 + where: + - variable: + name: MetaData/sensorScanPosition + is_in: 7-90 + action: + name: reduce obs space + + # Data Thinning + - filter: Gaussian Thinning + horizontal_mesh: 145 + use_reduced_horizontal_grid: true + distance_norm: geodesic + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + action: + name: reduce obs space + + # ------------------------ + obs prior filters: + # Zero Atmospheric clouds in CRTM where water_area_fraction < 0.99 + - filter: Variable Assignment + assignments: + - name: MetaData/zeroCloudInCRTM + type: int + function: + name: ObsFunction/Conditional + options: +# firstmatchingcase: false + defaultvalue: 0 # Will not zero clouds by default + cases: + - where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + value: 1 # Will zero clouds by default + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Step 0-B: Calculate derived variables + # Calculate CLW retrieved from observation + - filter: Variable Assignment + assignments: + - name: CLWRetFromObs@DerivedMetaData + type: float + function: + name: CLWRetMW@ObsFunction + options: + clwret_ch238: 1 + clwret_ch314: 2 + clwret_types: [ObsValue] + + # Calculate CLW retrieved from observation + - filter: Variable Assignment + assignments: + - name: CLWRetFromBkg@DerivedMetaData + type: float + function: + name: CLWRetMW@ObsFunction + options: + clwret_ch238: 1 + clwret_ch314: 2 + clwret_types: [HofX] + + # Calculate symmetric retrieved CLW + - filter: Variable Assignment + assignments: + - name: CLWRetSymmetric@DerivedMetaData + type: float + value: 1000.0 + + - filter: Variable Assignment + where: + - variable: + name: CLWRetFromObs@DerivedMetaData + minvalue: 0. + maxvalue: 999. + - variable: + name: CLWRetFromBkg@DerivedMetaData + minvalue: 0. + maxvalue: 999. + where operator: and + assignments: + - name: CLWRetSymmetric@DerivedMetaData + type: float + function: + name: Arithmetic@ObsFunction + options: + variables: + - name: CLWRetFromObs@DerivedMetaData + - name: CLWRetFromBkg@DerivedMetaData + total coefficient: 0.5 + + # Calculate scattering index from observation + - filter: Variable Assignment + assignments: + - name: SIRetFromObs@DerivedMetaData + type: float + function: + name: SCATRetMW@ObsFunction + options: + scatret_ch238: 1 + scatret_ch314: 2 + scatret_ch890: 16 + scatret_types: [ObsValue] + + # Calculate CLW obs/bkg match index + - filter: Variable Assignment + assignments: + - name: CLWMatchIndex@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: CLWMatchIndexMW@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + clwobs_function: + name: CLWRetFromObs@DerivedMetaData + clwbkg_function: + name: CLWRetFromBkg@DerivedMetaData + clwret_clearsky: [0.030, 0.030, 0.030, 0.020, 0.030, 0.080, 0.150, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.020, 0.030, 0.030, + 0.030, 0.030, 0.050, 0.100] + + # Calculate symmetric observation error + - filter: Variable Assignment + assignments: + - name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorModelRamp@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + xvar: + name: CLWRetSymmetric@DerivedMetaData + x0: [0.030, 0.030, 0.030, 0.020, 0.030, 0.080, 0.150, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.000, 0.020, 0.030, 0.030, 0.030, 0.030, + 0.050, 0.100] + x1: [0.350, 0.380, 0.400, 0.450, 0.500, 1.000, 1.000, 0.000, 0.000, 0.000, + 0.000, 0.000, 0.000, 0.000, 0.000, 0.350, 0.500, 0.500, 0.500, 0.500, + 0.500, 0.500] + err0: [4.500, 4.500, 4.500, 2.500, 0.550, 0.300, 0.300, 0.400, 0.400, 0.400, + 0.450, 0.450, 0.550, 0.800, 4.000, 4.000, 4.000, 3.500, 3.000, 3.000, + 3.000, 3.000] + err1: [20.000, 25.000, 12.000, 7.000, 3.500, 3.000, 0.800, 0.400, 0.400, + 0.400, 0.450, 0.450, 0.550, 0.800, 4.000, 19.000, 30.000, 25.000, 16.500, + 12.000, 9.000, 6.500] + + # Calculate Innovation@DerivedMetaData + - filter: Variable Assignment + assignments: + - name: Innovation@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsFunction/Arithmetic + channels: *{{observation_from_jcb}}_simulated_channels + options: + variables: + - name: brightnessTemperature@ObsValue + channels: *{{observation_from_jcb}}_simulated_channels + - name: brightnessTemperature@HofX + channels: *{{observation_from_jcb}}_simulated_channels + coefs: [1, -1] + + # Step 0-C: Assign Initial All-Sky Observation Error + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: assign error + error function: + name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 1A: CLW Retrieval Check (observation_based) + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: 1-7, 16-22 + test variables: + - name: CLWRetFromObs@DerivedMetaData + maxvalue: 999.0 + action: + name: reject + + # Step 1B: CLW Retrieval Check (background_based) + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: 1-7, 16-22 + test variables: + - name: CLWRetFromBkg@DerivedMetaData + maxvalue: 999.0 + action: + name: reject + + # Step 2: Window Channel Sanity Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: 1-7, 16, 17-22 + test variables: + - name: Innovation@DerivedMetaData + channels: 1, 2, 5-7, 16 + maxvalue: 200.0 + minvalue: -200.0 + flag all filter variables if any test variable is out of bounds: true + action: + name: reject + + # Step 3: Hydrometeor Check (cloud/precipitation affected chanels) + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/HydrometeorCheckATMS + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: HydrometeorCheckATMS@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_clearsky: [4.500, 4.500, 4.500, 2.500, 0.550, 0.300, 0.300, 0.400, + 0.400, 0.400, 0.450, 0.450, 0.550, 0.800, 4.000, 4.000, 4.000, 3.500, + 3.000, 3.000, 3.000, 3.000] + clwret_function: + name: CLWRetFromObs@DerivedMetaData + obserr_function: + name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + Clouds: *{{observation_from_jcb}}_clouds + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: DerivedMetaData/HydrometeorCheckATMS + channels: *{{observation_from_jcb}}_simulated_channels + maxvalue: 0.0 + action: + name: reject + + # Step 4: Observation Error Inflation based on Topography Check + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorTopo@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorFactorTopoRad@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsErrorFactorTopo@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 5: Obs Error Inflation based on TOA Transmittancec Check + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorTransmitTop@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorFactorTransmitTopRad@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsErrorFactorTransmitTop@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 6: Observation Error Inflation based on Surface Jacobian Check + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorSurfJacobian@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorFactorSurfJacobianRad@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + obserr_demisf: [0.010, 0.020, 0.015, 0.020, 0.200] + obserr_dtempf: [0.500, 2.000, 1.000, 2.000, 4.500] + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsErrorFactorSurfJacobian@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 7: Situation Dependent Check + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorSituDepend@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorFactorSituDependMW@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + clwbkg_function: + name: CLWRetFromBkg@DerivedMetaData + clwobs_function: + name: CLWRetFromObs@DerivedMetaData + scatobs_function: + name: SIRetFromObs@DerivedMetaData + clwmatchidx_function: + name: CLWMatchIndex@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + obserr_function: + name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + obserr_clearsky: [4.500, 4.500, 4.500, 2.500, 0.550, 0.300, 0.300, 0.400, + 0.400, 0.400, 0.450, 0.450, 0.550, 0.800, 4.000, 4.000, 4.000, 3.500, + 3.000, 3.000, 3.000, 3.000] + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsErrorFactorSituDepend@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 8: Gross check + # Remove data if abs(Obs-HofX) > absolute threshold + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorLat@DerivedMetaData + type: float + function: + name: ObsErrorFactorLatRad@ObsFunction + options: + latitude_parameters: [25.0, 0.25, 0.04, 3.0] + + - filter: Variable Assignment + assignments: + - name: ObsErrorBound@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsErrorBoundMW@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_latitude: + name: ObsErrorFactorLat@DerivedMetaData + obserr_bound_transmittop: + name: ObsErrorFactorTransmitTop@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_topo: + name: ObsErrorFactorTopo@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + obserr_function: + name: InitialObsError@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + threshold: 3 + threshold_precip: 2.5 + obserr_bound_max: [4.5, 4.5, 3.0, 3.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, + 1.0, 1.0, 2.0, 4.5, 4.5, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0] + + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + function absolute threshold: + - name: ObsErrorBound@DerivedMetaData + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: reject + + # Step 9: Inter-Channel Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: InterChannelConsistencyCheck@ObsFunction + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use passive_bc: true + sensor: *{{observation_from_jcb}}_sensor_id + use_flag: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 4, 1, 1, 1, 1, 1, 1, + 1] + maxvalue: 1.0e-12 + action: + name: reject + + # Step 10: Useflag Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/ChannelUseflagCheckRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 4, 1, 1, 1, 1, 1, 1, + 1] + minvalue: 1.0e-12 + action: + name: reject + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnLiquidIceCloud + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: GeoVaLs/mass_content_of_cloud_liquid_water_in_atmosphere_column + - name: GeoVaLs/mass_content_of_cloud_ice_in_atmosphere_column + coefs: [1, 1] +# replace zero by 9.99e-7 in DerivedMetaData/ColumnLiquidIceCloud + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnLiquidIceCloud + type: float + value: 9.99e-07 + where: + - variable: + name: DerivedMetaData/ColumnLiquidIceCloud + maxvalue: 9.99e-07 + +# ratio of liquid /(Liquid+Ice) + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnRatioLiquidCloud + type: float + function: + name: ObsFunction/ElementMultiply + options: + variables: + - name: GeoVaLs/mass_content_of_cloud_liquid_water_in_atmosphere_column + - name: DerivedMetaData/ColumnLiquidIceCloud + exponents: [1,-1] + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnRatioLiquidCloud + type: float + value: 0.0 + where: + - variable: + name: DerivedMetaData/ColumnLiquidIceCloud + maxvalue: 1.0e-06 +# TotalColumnVaporGuess + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/TotalColumnVaporGuess + type: float + function: + name: ObsFunction/TotalColumnVaporGuess +# potentialTemperature at surface + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureSurface + type: float + function: + name: PotentialTemperatureFromTemperature@ObsFunction + options: + use surface pressure: true +# potentialTemperature near 700 hPa + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureAt700hPa + type: float + function: + name: PotentialTemperatureFromTemperature@ObsFunction + options: + use surface pressure: false + pressure to evaluate potential temperature: 70000.0 +# stability = Difference between PotentialTemperatur values + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureDiff + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: DerivedMetaData/PotentialTemperatureAt700hPa + - name: DerivedMetaData/PotentialTemperatureSurface + coefs: [1, -1] +# cold-air-outbreak (cao) check for all-sky DA with Rain, Snow, or Graupel + - filter: BlackList + filter variables: + - name: brightnessTemperature + channels: 1-7, 16-22 + where: + - variable: + name: ObsFunction/TotalColumnVaporGuess + maxvalue: 8.0 + - variable: + name: DerivedMetaData/ColumnRatioLiquidCloud + maxvalue: 0.5 + - variable: + name: DerivedMetaData/PotentialTemperatureDiff + maxvalue: 12.0 + - variable: + name: GeoVaLs/water_area_fraction + minvalue: 0.99 + action: + name: reject + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 index d18985cd8..d1e8fb70b 100644 --- a/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 @@ -22,8 +22,11 @@ obs operator: name: CRTM Absorbers: [H2O, O3, CO2] - Clouds: [Water, Ice] - Cloud_Fraction: 1.0 + Clouds: &{{observation_from_jcb}}_clouds [Water, Ice, Rain, Snow, Graupel] +# Cloud_Fraction: 1.0 +# ---methods for cloud fraction and radii in fov: thompson, or none + method for cloud fraction within fov: thompson + method for hydrometeor effective radii within fov: thompson Cloud_Seeding: true obs options: Sensor_ID: &{{observation_from_jcb}}_sensor_id atms_npp @@ -37,6 +40,8 @@ obs bias: input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" + variables without bc: [brightnessTemperature] + channels: {{ get_satellite_variable(observation_from_jcb, "not_biascorrtd") }} variational bc: predictors: - name: constant @@ -65,39 +70,48 @@ ratio for small dataset: 2.0 output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" - # Observation Pre Filters (QC) - # ---------------------------- + # ------------------------ obs pre filters: - # Step 0-A: Create Diagnostic Flags - - filter: Create Diagnostic Flags + # Remove Observations from the Edge of the Scan + - filter: Domain Check filter variables: - name: brightnessTemperature - channels: *{{observation_from_jcb}}_simulated_channels - flags: - - name: ScanEdgeRemoval - initial value: false - force reinitialization: false - - name: Thinning - initial value: false - force reinitialization: false - - name: CLWRetrievalCheck - initial value: false - force reinitialization: false - - name: WindowChannelExtremeResidual - initial value: false - force reinitialization: false - - name: HydrometeorCheck - initial value: false - force reinitialization: false - - name: GrossCheck - initial value: false - force reinitialization: false - - name: InterChannelConsistency - initial value: false - force reinitialization: false - - name: UseflagCheck - initial value: false - force reinitialization: false + channels: 1-22 + where: + - variable: + name: MetaData/sensorScanPosition + is_in: 7-90 + action: + name: reduce obs space + + # Data Thinning + - filter: Gaussian Thinning + horizontal_mesh: 145 + use_reduced_horizontal_grid: true + distance_norm: geodesic + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + action: + name: reduce obs space + + # ------------------------ + obs prior filters: + # Zero Atmospheric clouds in CRTM where water_area_fraction < 0.99 + - filter: Variable Assignment + assignments: + - name: MetaData/zeroCloudInCRTM + type: int + function: + name: ObsFunction/Conditional + options: +# firstmatchingcase: false + defaultvalue: 0 # Will not zero clouds by default + cases: + - where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + value: 1 # Will zero clouds by default # Observation Post Filters (QC) # ----------------------------- @@ -242,33 +256,7 @@ name: InitialObsError@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels - # Step 1: Remove Observations from the Edge of the Scan - - filter: Domain Check - filter variables: - - name: brightnessTemperature - channels: 1-22 - where: - - variable: - name: MetaData/sensorScanPosition - is_in: 7-90 - actions: - - name: set - flag: ScanEdgeRemoval - - name: reject - - # Step 2: Data Thinning - - filter: Gaussian Thinning - horizontal_mesh: 145 - use_reduced_horizontal_grid: true - distance_norm: geodesic - # round_horizontal_bin_count_to_nearest: true - # partition_longitude_bins_using_mesh: true - actions: - - name: set - flag: Thinning - - name: reject - - # Step 3A: CLW Retrieval Check (observation_based) + # Step 1A: CLW Retrieval Check (observation_based) - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -276,12 +264,10 @@ test variables: - name: CLWRetFromObs@DerivedMetaData maxvalue: 999.0 - actions: - - name: set - flag: CLWRetrievalCheck - - name: reject + action: + name: reject - # Step 3B: CLW Retrieval Check (background_based) + # Step 1B: CLW Retrieval Check (background_based) - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -289,12 +275,10 @@ test variables: - name: CLWRetFromBkg@DerivedMetaData maxvalue: 999.0 - actions: - - name: set - flag: CLWRetrievalCheck - - name: reject + action: + name: reject - # Step 4: Window Channel Sanity Check + # Step 2: Window Channel Sanity Check - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -305,12 +289,10 @@ maxvalue: 200.0 minvalue: -200.0 flag all filter variables if any test variable is out of bounds: true - actions: - - name: set - flag: WindowChannelExtremeResidual - - name: reject + action: + name: reject - # Step 5: Hydrometeor Check (cloud/precipitation affected chanels) + # Step 3: Hydrometeor Check (cloud/precipitation affected chanels) - filter: Variable Assignment assignments: - name: DerivedMetaData/HydrometeorCheckATMS @@ -329,7 +311,7 @@ obserr_function: name: InitialObsError@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels - + Clouds: *{{observation_from_jcb}}_clouds - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -338,13 +320,10 @@ - name: DerivedMetaData/HydrometeorCheckATMS channels: *{{observation_from_jcb}}_simulated_channels maxvalue: 0.0 - actions: - - name: set - flag: HydrometeorCheck - ignore: rejected observations - - name: reject + action: + name: reject - # Step 6: Observation Error Inflation based on Topography Check + # Step 4: Observation Error Inflation based on Topography Check - filter: Variable Assignment assignments: - name: ObsErrorFactorTopo@DerivedMetaData @@ -367,7 +346,7 @@ name: ObsErrorFactorTopo@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels - # Step 7: Obs Error Inflation based on TOA Transmittancec Check + # Step 5: Obs Error Inflation based on TOA Transmittancec Check - filter: Variable Assignment assignments: - name: ObsErrorFactorTransmitTop@DerivedMetaData @@ -389,7 +368,7 @@ name: ObsErrorFactorTransmitTop@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels - # Step 8: Observation Error Inflation based on Surface Jacobian Check + # Step 6: Observation Error Inflation based on Surface Jacobian Check - filter: Variable Assignment assignments: - name: ObsErrorFactorSurfJacobian@DerivedMetaData @@ -414,7 +393,7 @@ name: ObsErrorFactorSurfJacobian@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels - # Step 9: Situation Dependent Check + # Step 7: Situation Dependent Check - filter: Variable Assignment assignments: - name: ObsErrorFactorSituDepend@DerivedMetaData @@ -452,8 +431,8 @@ name: ObsErrorFactorSituDepend@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels - # Step 10: Gross check - # Remove data if abs(Obs-HofX) > absolute threhold + # Step 8: Gross check + # Remove data if abs(Obs-HofX) > absolute threshold - filter: Variable Assignment assignments: - name: ObsErrorFactorLat@DerivedMetaData @@ -487,7 +466,8 @@ obserr_function: name: InitialObsError@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels - threhold: 3 + threshold: 3 + threshold_precip: 2.5 obserr_bound_max: [4.5, 4.5, 3.0, 3.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 2.0, 4.5, 4.5, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0] @@ -498,13 +478,10 @@ function absolute threshold: - name: ObsErrorBound@DerivedMetaData channels: *{{observation_from_jcb}}_simulated_channels - actions: - - name: set - flag: GrossCheck - ignore: rejected observations - - name: reject + action: + name: reject - # Step 11: Inter-Channel Check + # Step 9: Inter-Channel Check - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -519,13 +496,10 @@ use_flag: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 4, 1, 1, 1, 1, 1, 1, 1] maxvalue: 1.0e-12 - actions: - - name: set - flag: InterChannelConsistency - ignore: rejected observations - - name: reject + action: + name: reject - # Step 12: Useflag Check + # Step 10: Useflag Check - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -538,13 +512,109 @@ use_flag: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 4, 1, 1, 1, 1, 1, 1, 1] minvalue: 1.0e-12 - actions: - - name: set - flag: UseflagCheck - ignore: rejected observations - - name: reject - + action: + name: reject + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnLiquidIceCloud + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: GeoVaLs/mass_content_of_cloud_liquid_water_in_atmosphere_column + - name: GeoVaLs/mass_content_of_cloud_ice_in_atmosphere_column + coefs: [1, 1] +# replace zero by 9.99e-7 in DerivedMetaData/ColumnLiquidIceCloud + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnLiquidIceCloud + type: float + value: 9.99e-07 + where: + - variable: + name: DerivedMetaData/ColumnLiquidIceCloud + maxvalue: 9.99e-07 +# ratio of liquid /(Liquid+Ice) + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnRatioLiquidCloud + type: float + function: + name: ObsFunction/ElementMultiply + options: + variables: + - name: GeoVaLs/mass_content_of_cloud_liquid_water_in_atmosphere_column + - name: DerivedMetaData/ColumnLiquidIceCloud + exponents: [1,-1] + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnRatioLiquidCloud + type: float + value: 0.0 + where: + - variable: + name: DerivedMetaData/ColumnLiquidIceCloud + maxvalue: 1.0e-06 +# TotalColumnVaporGuess + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/TotalColumnVaporGuess + type: float + function: + name: ObsFunction/TotalColumnVaporGuess +# potentialTemperature at surface + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureSurface + type: float + function: + name: PotentialTemperatureFromTemperature@ObsFunction + options: + use surface pressure: true +# potentialTemperature near 700 hPa + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureAt700hPa + type: float + function: + name: PotentialTemperatureFromTemperature@ObsFunction + options: + use surface pressure: false + pressure to evaluate potential temperature: 70000.0 +# stability = Difference between PotentialTemperatur values + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureDiff + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: DerivedMetaData/PotentialTemperatureAt700hPa + - name: DerivedMetaData/PotentialTemperatureSurface + coefs: [1, -1] +# cold-air-outbreak (cao) check for all-sky DA with Rain, Snow, or Graupel + - filter: BlackList + filter variables: + - name: brightnessTemperature + channels: 1-7, 16-22 + where: + - variable: + name: ObsFunction/TotalColumnVaporGuess + maxvalue: 8.0 + - variable: + name: DerivedMetaData/ColumnRatioLiquidCloud + maxvalue: 0.5 + - variable: + name: DerivedMetaData/PotentialTemperatureDiff + maxvalue: 12.0 + - variable: + name: GeoVaLs/water_area_fraction + minvalue: 0.99 + action: + name: reject # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: From 48137bb54839eb2fe776403069cefa3dc6795b5c Mon Sep 17 00:00:00 2001 From: emilyhcliu <36091766+emilyhcliu@users.noreply.github.com> Date: Mon, 7 Apr 2025 10:29:18 -0400 Subject: [PATCH 103/199] Minor changes for IASI to use jcb. Does not affect results (#99) Some parameters in the IASI YAML have not been queried from JCB yet. So, this PR added the specification of parameters from JCB. Minor changes do not change results. --- .../atmosphere/iasi_metop-a.yaml.j2 | 235 +----------------- .../atmosphere/iasi_metop-b.yaml.j2 | 235 +----------------- 2 files changed, 18 insertions(+), 452 deletions(-) diff --git a/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 index 890e2705d..ab41e565d 100644 --- a/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 @@ -104,54 +104,7 @@ channels: *{{observation_from_jcb}}_simulated_channels action: name: assign error - error parameter vector: &{{observation_from_jcb}}_oberr [1.38, 0.81, 0.75, 0.79, 0.72, 0.74, - 0.68, 0.72, 0.65, 0.65, 0.65, 0.69, 0.64, 0.64, 0.65, 0.67, 0.62, 0.61, 0.62, - 0.64, 0.59, 0.76, 1.22, 0.78, 0.64, 0.62, 0.61, 0.69, 0.65, 0.59, 0.61, 0.59, - 0.68, 0.62, 0.68, 4.38, 3.05, 2.31, 1.56, 1.33, 1.58, 0.93, 1.67, 0.72, 0.57, - 0.58, 0.55, 0.68, 0.59, 0.68, 0.59, 0.65, 0.58, 0.62, 0.64, 0.58, 0.64, 0.55, - 0.64, 0.5, 0.82, 0.59, 0.62, 0.51, 0.64, 0.52, 0.51, 0.51, 0.76, 0.52, 0.57, - 0.55, 0.69, 0.58, 0.65, 0.61, 0.59, 0.64, 0.76, 0.72, 1.05, 0.75, 0.51, 0.65, - 1.3, 0.69, 0.93, 1.49, 1.12, 0.68, 0.66, 0.67, 0.59, 0.59, 0.69, 0.67, 0.64, - 0.62, 0.72, 0.69, 0.66, 0.79, 0.78, 0.74, 0.88, 0.77, 0.88, 0.86, 1, 0.87, - 0.85, 0.88, 0.84, 0.84, 0.84, 0.8, 0.8, 0.87, 0.98, 0.52, 0.65, 0.69, 0.61, - 0.6, 0.67, 0.79, 0.62, 0.66, 0.7, 0.65, 0.62, 0.61, 0.62, 0.53, 0.6, 0.68, - 0.95, 0.63, 0.97, 0.65, 0.98, 0.58, 0.73, 0.65, 0.85, 0.99, 0.76, 0.85, 0.97, - 0.77, 0.62, 0.63, 1.21, 1.41, 1.55, 1.78, 1.35, 1.14, 1.69, 1.79, 1.46, 1.63, - 1.94, 2.01, 1.24, 1.76, 1.26, 1.47, 1.9, 1.66, 2.13, 1.49, 1.52, 1.55, 1.96, - 2.31, 2.33, 2.32, 2.31, 2.33, 2.23, 2.33, 1.84, 2.29, 2.28, 2.28, 2.28, 2.26, - 2.26, 2.26, 2.27, 2.24, 2.23, 2.24, 2.26, 2.28, 2.28, 2.3, 2.15, 2.31, 2.37, - 2.27, 2.29, 2.29, 2.23, 2.28, 2.32, 2.32, 2.31, 2.32, 2.32, 2.31, 2.31, 2.28, - 2.29, 2.28, 2.26, 2.29, 2.27, 2.26, 2.25, 2.27, 2.24, 2.21, 2.24, 2.17, 2.18, - 2.17, 2.21, 1.99, 2.16, 2.2, 2.13, 2.12, 2.13, 2.1, 2.12, 2.11, 2.09, 2.09, - 2.08, 2.09, 2.04, 2.04, 2.1, 2.01, 2.05, 2.03, 2.06, 1.98, 1.95, 1.94, 1.91, - 1.7, 1.76, 1.77, 1.83, 2.04, 1.91, 1.99, 1.99, 2.07, 2.02, 2.04, 2.1, 2.06, - 2.18, 2.21, 2.24, 2.23, 2.23, 1.98, 2.2, 2.18, 2.18, 2.21, 2.23, 2.24, 2.24, - 2.25, 1.8, 2.24, 1.73, 1.73, 2.27, 1.67, 2.21, 1.72, 2.23, 2.23, 2.23, 2.24, - 2.23, 2.12, 2.17, 1.74, 2.02, 1.88, 1.67, 1.73, 1.83, 1.82, 1.73, 1.83, 2.19, - 1.84, 1.89, 1.6, 1.71, 1.86, 1.85, 1.84, 1.87, 1.91, 1.52, 1.95, 1.87, 1.89, - 1.91, 1.91, 1.93, 1.9, 1.91, 1.9, 1.89, 1.89, 1.91, 1.9, 1.91, 1.91, 1.91, - 1.93, 1.94, 1.91, 1.92, 1.77, 1.91, 1.95, 1.19, 1.96, 1.98, 1.94, 1.55, 1.91, - 1.92, 1.92, 1.97, 1.93, 1.99, 1.86, 1.12, 1.93, 1.92, 1.95, 1.85, 1.84, 1.91, - 1.12, 1.82, 1.82, 1.95, 1.24, 1.94, 1.96, 1.21, 1.83, 1.96, 1.36, 1.96, 1.82, - 1.92, 1.68, 1.93, 1.23, 1.96, 1.93, 1.86, 1.41, 1.16, 1.6, 1.25, 1.2, 1.65, - 1.66, 1.87, 1.94, 1.96, 1.91, 1.25, 1.93, 1.91, 1.7, 0.99, 1.81, 1.92, 1.95, - 1.5, 1.47, 1.15, 1.58, 1.18, 1.82, 1.13, 1.83, 1.91, 1.26, 1.27, 1.91, 1.45, - 1.6, 1.29, 1.94, 1.94, 1.23, 1.95, 1.21, 1.94, 1.86, 1.9, 1.33, 1.75, 2.02, - 1.98, 2.03, 1.83, 1.5, 2.04, 2.02, 1.9, 2, 2.02, 1.95, 1.93, 1.95, 1.95, 1.99, - 2, 1.94, 1.96, 1.86, 1.92, 1.88, 1.86, 1.84, 1.87, 1.77, 1.89, 1.89, 1.88, - 1.94, 1.82, 1.79, 1.86, 2.06, 2.33, 1.88, 1.86, 1.81, 1.8, 1.8, 1.86, 1.9, - 2, 2.06, 2.1, 2.2, 2, 2.16, 1.98, 1.8, 1.8, 1.85, 1.75, 2.04, 2.19, 2.14, - 2.19, 1.86, 2.1, 2.11, 2.18, 2.03, 2.28, 2.19, 2.26, 2.26, 2.21, 2.21, 2.26, - 2.33, 2.27, 2.21, 2.12, 2.23, 2.26, 2.25, 1.88, 2.26, 2.24, 2.36, 2.29, 2.35, - 2.3, 2.27, 2.08, 2.05, 2.27, 2.28, 2.27, 2.28, 1.97, 2.25, 2.25, 2.25, 2.31, - 2.28, 2.27, 2.13, 2.24, 2.28, 2.28, 2.41, 2.34, 9.32, 2.28, 2.38, 2.27, 2.27, - 2.39, 2.11, 2.09, 2.1, 2.06, 2.12, 2.08, 2, 1.93, 2.02, 2.55, 1.54, 1.64, - 1.51, 1.55, 2.82, 2.92, 2.55, 2.37, 1.85, 1.6, 1.72, 1.74, 1.79, 1.9, 1.94, - 2, 2.04, 2.08, 2.12, 2.13, 2.16, 2.18, 2.18, 2.2, 2.2, 2.41, 2.39, 2.38, 2.4, - 2.42, 2.41, 2.43, 2.45, 2.43, 2.45, 2.43, 2.4, 2.44, 2.4, 2.42, 2.43, 2.45, - 2.45, 2.45, 2.46, 2.45, 2.45, 2.43, 2.51, 2.48, 2.48, 2.53, 2.46, 2.49, 2.5, - 2.5, 2.5, 2.52, 2.52, 2.54, 2.5, 2.48, 2.5, 2.55, 2.5, 2.48, 2.5, 2.5, 2.52, - 2.52, 2.48, 2.5, 2.5, 2.52, 2.46, 2.53, 9] + error parameter vector: &{{observation_from_jcb}}_oberr [{{ get_satellite_variable(observation_from_jcb, "error") }}] # Step 4: Remove Observations from the Edge of the Scan - filter: Domain Check @@ -163,7 +116,6 @@ name: MetaData/sensorScanPosition is_in: 5-56 action: - name: reject name: reduce obs space # Step 5: Data Thinning @@ -175,10 +127,7 @@ use_reduced_horizontal_grid: true distance_norm: geodesic priority_variable: MetaData/fractionOfClearPixelsInFOV - # round_horizontal_bin_count_to_nearest: true - # partition_longitude_bins_using_mesh: true action: - name: reject name: reduce obs space # Observation Post Filters (QC) @@ -198,7 +147,7 @@ name: MetaData/solarZenithAngle maxvalue: 88.9999 - variable: - name: water_area_fraction@GeoVaLs + name: GeoVaLs/water_area_fraction minvalue: 1.0e-12 action: name: reject @@ -211,7 +160,7 @@ action: name: inflate error inflation variable: - name: ObsErrorFactorWavenumIR@ObsFunction + name: ObsFunction/ObsErrorFactorWavenumIR channels: *{{observation_from_jcb}}_simulated_channels options: channels: *{{observation_from_jcb}}_simulated_channels @@ -264,71 +213,8 @@ channels: *{{observation_from_jcb}}_simulated_channels options: channels: *{{observation_from_jcb}}_simulated_channels - use_flag: [1, -1, -1, -1, 1, -1, -1, -1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, - 1, 1, -1, 1, 1, 1, -1, -1, 1, 1, -1, 1, -1, 1, -1, 1, -1, -1, -1, -1, -1, - -1, -1, -1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, -1, - 1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, -1, -1, - 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, 1, 1, 1, - 1, 1, 1, -1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, 1, 1, -1, -1, -1, - 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, - 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, -1, 1, 1, 1, -1, 1, 1, -1, 1, -1, - 1, -1, -1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, -1, -1, -1, 1, 1, 1, -1, - -1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, 1, -1, 1, 1, -1, 1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, - -1, -1, 1, -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, 1, - -1, 1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, -1, -1, -1, -1, -1, 1, -1, 1, -1, - 1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, - -1, -1, 1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, - -1, -1, -1, -1, -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, 1, - -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1] - use_flag_clddet: [1, -1, -1, -1, 1, -1, -1, -1, 1, -1, 1, -1, 1, -1, 1, 1, - 1, -1, 1, 1, -1, 1, 1, 1, -1, -1, 1, 1, -1, 1, -1, 1, -1, 1, -1, -1, -1, - -1, -1, -1, -1, -1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, - 1, 1, -1, 1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, - -1, -1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, 1, - 1, 1, 1, 1, 1, -1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, 1, 1, -1, - -1, -1, 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, 1, - 1, 1, 1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, -1, 1, 1, 1, -1, 1, 1, - -1, 1, -1, 1, -1, -1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, -1, -1, -1, 1, - 1, 1, -1, -1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, 1, -1, 1, 1, -1, 1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, 1, -1, 1, -1, 1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, - -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1] + use_flag: &{{observation_from_jcb}}_active_channels [{{ get_satellite_variable(observation_from_jcb, "active") }}] + use_flag_clddet: [{{ get_satellite_variable(observation_from_jcb, "icld_det") }}] obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] error parameter vector: *{{observation_from_jcb}}_oberr maxvalue: 1.0e-12 @@ -341,42 +227,11 @@ - name: brightnessTemperature channels: *{{observation_from_jcb}}_simulated_channels test variables: - - name: NearSSTRetCheckIR@ObsFunction + - name: ObsFunction/NearSSTRetCheckIR channels: *{{observation_from_jcb}}_simulated_channels options: channels: *{{observation_from_jcb}}_simulated_channels - use_flag: [1, -1, -1, -1, 1, -1, -1, -1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, - 1, 1, -1, 1, 1, 1, -1, -1, 1, 1, -1, 1, -1, 1, -1, 1, -1, -1, -1, -1, -1, - -1, -1, -1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, -1, - 1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, -1, -1, - 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, 1, 1, 1, - 1, 1, 1, -1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, 1, 1, -1, -1, -1, - 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, - 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, -1, 1, 1, 1, -1, 1, 1, -1, 1, -1, - 1, -1, -1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, -1, -1, -1, 1, 1, 1, -1, - -1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, 1, -1, 1, 1, -1, 1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, - -1, -1, 1, -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, 1, - -1, 1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, -1, -1, -1, -1, -1, 1, -1, 1, -1, - 1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, - -1, -1, 1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, - -1, -1, -1, -1, -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, 1, - -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1] + use_flag: *{{observation_from_jcb}}_active_channels obserr_demisf: [0.01, 0.02, 0.03, 0.02, 0.03] obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] maxvalue: 1.0e-12 @@ -418,48 +273,7 @@ channels: *{{observation_from_jcb}}_simulated_channels options: channels: *{{observation_from_jcb}}_simulated_channels - obserr_bound_max: [3.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 4.0, 2.0, 2.0, 2.0, - 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 4.0, 4.0, 3.5, - 2.5, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 3.5, 2.0, 2.0, 2.0, 2.0, 2.0, - 2.0, 3.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - 2.0, 2.0, 2.0, 2.5, 2.0, 2.5, 2.5, 3.0, 2.5, 2.5, 2.5, 2.5, 3.5, 2.5, 2.5, - 3.0, 3.5, 3.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.5, 4.5, 4.5, 4.5, 4.5, 4.0, - 4.5, 4.0, 4.0, 4.5, 2.5, 3.0, 2.5, 3.0, 2.5, 3.0, 2.0, 2.5, 2.5, 3.0, 3.0, - 2.5, 3.0, 3.0, 3.0, 2.5, 2.5, 4.0, 4.5, 4.5, 5.0, 4.0, 4.0, 5.0, 5.0, 5.0, - 5.0, 5.5, 5.5, 4.0, 5.0, 4.0, 4.5, 5.5, 5.5, 6.0, 4.5, 4.5, 4.0, 5.0, 5.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 5.5, 4.5, 6.0, 5.0, 5.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 5.0, 6.0, 6.0, 6.0, 4.0, 6.0, 6.0, 6.0, 6.0, 4.5, 6.0, 6.0, 4.5, - 6.0, 6.0, 6.0, 6.0, 6.0, 5.0, 6.0, 6.0, 6.0, 5.0, 6.0, 6.0, 5.0, 6.0, 5.0, - 6.0, 6.0, 6.0, 5.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0] + obserr_bound_max: [{{ get_satellite_variable(observation_from_jcb, "ermax") }}] error parameter vector: *{{observation_from_jcb}}_oberr action: name: reject @@ -474,38 +288,7 @@ channels: *{{observation_from_jcb}}_simulated_channels options: channels: *{{observation_from_jcb}}_simulated_channels - use_flag: [1, -1, -1, -1, 1, -1, -1, -1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, - 1, 1, -1, 1, 1, 1, -1, -1, 1, 1, -1, 1, -1, 1, -1, 1, -1, -1, -1, -1, -1, - -1, -1, -1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, -1, - 1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, -1, -1, - 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, 1, 1, 1, - 1, 1, 1, -1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, 1, 1, -1, -1, -1, - 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, - 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, -1, 1, 1, 1, -1, 1, 1, -1, 1, -1, - 1, -1, -1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, -1, -1, -1, 1, 1, 1, -1, - -1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, 1, -1, 1, 1, -1, 1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, - -1, -1, 1, -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, 1, - -1, 1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, -1, -1, -1, -1, -1, 1, -1, 1, -1, - 1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, - -1, -1, 1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, - -1, -1, -1, -1, -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, 1, - -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1] + use_flag: *{{observation_from_jcb}}_active_channels minvalue: 1.0e-12 action: name: reject diff --git a/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 index 57e9236b3..1490345eb 100644 --- a/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 @@ -104,54 +104,7 @@ channels: *{{observation_from_jcb}}_simulated_channels action: name: assign error - error parameter vector: &{{observation_from_jcb}}_oberr [1.38, 0.81, 0.75, 0.79, 0.72, 0.74, - 0.68, 0.72, 0.65, 0.65, 0.65, 0.69, 0.64, 0.64, 0.65, 0.67, 0.62, 0.61, 0.62, - 0.64, 0.59, 0.76, 1.22, 0.78, 0.64, 0.62, 0.61, 0.69, 0.65, 0.59, 0.61, 0.59, - 0.68, 0.62, 0.68, 4.38, 3.05, 2.31, 1.56, 1.33, 1.58, 0.93, 1.67, 0.72, 0.57, - 0.58, 0.55, 0.68, 0.59, 0.68, 0.59, 0.65, 0.58, 0.62, 0.64, 0.58, 0.64, 0.55, - 0.64, 0.5, 0.82, 0.59, 0.62, 0.51, 0.64, 0.52, 0.51, 0.51, 0.76, 0.52, 0.57, - 0.55, 0.69, 0.58, 0.65, 0.61, 0.59, 0.64, 0.76, 0.72, 1.05, 0.75, 0.51, 0.65, - 1.3, 0.69, 0.93, 1.49, 1.12, 0.68, 0.66, 0.67, 0.59, 0.59, 0.69, 0.67, 0.64, - 0.62, 0.72, 0.69, 0.66, 0.79, 0.78, 0.74, 0.88, 0.77, 0.88, 0.86, 1, 0.87, - 0.85, 0.88, 0.84, 0.84, 0.84, 0.8, 0.8, 0.87, 0.98, 0.52, 0.65, 0.69, 0.61, - 0.6, 0.67, 0.79, 0.62, 0.66, 0.7, 0.65, 0.62, 0.61, 0.62, 0.53, 0.6, 0.68, - 0.95, 0.63, 0.97, 0.65, 0.98, 0.58, 0.73, 0.65, 0.85, 0.99, 0.76, 0.85, 0.97, - 0.77, 0.62, 0.63, 1.21, 1.41, 1.55, 1.78, 1.35, 1.14, 1.69, 1.79, 1.46, 1.63, - 1.94, 2.01, 1.24, 1.76, 1.26, 1.47, 1.9, 1.66, 2.13, 1.49, 1.52, 1.55, 1.96, - 2.31, 2.33, 2.32, 2.31, 2.33, 2.23, 2.33, 1.84, 2.29, 2.28, 2.28, 2.28, 2.26, - 2.26, 2.26, 2.27, 2.24, 2.23, 2.24, 2.26, 2.28, 2.28, 2.3, 2.15, 2.31, 2.37, - 2.27, 2.29, 2.29, 2.23, 2.28, 2.32, 2.32, 2.31, 2.32, 2.32, 2.31, 2.31, 2.28, - 2.29, 2.28, 2.26, 2.29, 2.27, 2.26, 2.25, 2.27, 2.24, 2.21, 2.24, 2.17, 2.18, - 2.17, 2.21, 1.99, 2.16, 2.2, 2.13, 2.12, 2.13, 2.1, 2.12, 2.11, 2.09, 2.09, - 2.08, 2.09, 2.04, 2.04, 2.1, 2.01, 2.05, 2.03, 2.06, 1.98, 1.95, 1.94, 1.91, - 1.7, 1.76, 1.77, 1.83, 2.04, 1.91, 1.99, 1.99, 2.07, 2.02, 2.04, 2.1, 2.06, - 2.18, 2.21, 2.24, 2.23, 2.23, 1.98, 2.2, 2.18, 2.18, 2.21, 2.23, 2.24, 2.24, - 2.25, 1.8, 2.24, 1.73, 1.73, 2.27, 1.67, 2.21, 1.72, 2.23, 2.23, 2.23, 2.24, - 2.23, 2.12, 2.17, 1.74, 2.02, 1.88, 1.67, 1.73, 1.83, 1.82, 1.73, 1.83, 2.19, - 1.84, 1.89, 1.6, 1.71, 1.86, 1.85, 1.84, 1.87, 1.91, 1.52, 1.95, 1.87, 1.89, - 1.91, 1.91, 1.93, 1.9, 1.91, 1.9, 1.89, 1.89, 1.91, 1.9, 1.91, 1.91, 1.91, - 1.93, 1.94, 1.91, 1.92, 1.77, 1.91, 1.95, 1.19, 1.96, 1.98, 1.94, 1.55, 1.91, - 1.92, 1.92, 1.97, 1.93, 1.99, 1.86, 1.12, 1.93, 1.92, 1.95, 1.85, 1.84, 1.91, - 1.12, 1.82, 1.82, 1.95, 1.24, 1.94, 1.96, 1.21, 1.83, 1.96, 1.36, 1.96, 1.82, - 1.92, 1.68, 1.93, 1.23, 1.96, 1.93, 1.86, 1.41, 1.16, 1.6, 1.25, 1.2, 1.65, - 1.66, 1.87, 1.94, 1.96, 1.91, 1.25, 1.93, 1.91, 1.7, 0.99, 1.81, 1.92, 1.95, - 1.5, 1.47, 1.15, 1.58, 1.18, 1.82, 1.13, 1.83, 1.91, 1.26, 1.27, 1.91, 1.45, - 1.6, 1.29, 1.94, 1.94, 1.23, 1.95, 1.21, 1.94, 1.86, 1.9, 1.33, 1.75, 2.02, - 1.98, 2.03, 1.83, 1.5, 2.04, 2.02, 1.9, 2, 2.02, 1.95, 1.93, 1.95, 1.95, 1.99, - 2, 1.94, 1.96, 1.86, 1.92, 1.88, 1.86, 1.84, 1.87, 1.77, 1.89, 1.89, 1.88, - 1.94, 1.82, 1.79, 1.86, 2.06, 2.33, 1.88, 1.86, 1.81, 1.8, 1.8, 1.86, 1.9, - 2, 2.06, 2.1, 2.2, 2, 2.16, 1.98, 1.8, 1.8, 1.85, 1.75, 2.04, 2.19, 2.14, - 2.19, 1.86, 2.1, 2.11, 2.18, 2.03, 2.28, 2.19, 2.26, 2.26, 2.21, 2.21, 2.26, - 2.33, 2.27, 2.21, 2.12, 2.23, 2.26, 2.25, 1.88, 2.26, 2.24, 2.36, 2.29, 2.35, - 2.3, 2.27, 2.08, 2.05, 2.27, 2.28, 2.27, 2.28, 1.97, 2.25, 2.25, 2.25, 2.31, - 2.28, 2.27, 2.13, 2.24, 2.28, 2.28, 2.41, 2.34, 9.32, 2.28, 2.38, 2.27, 2.27, - 2.39, 2.11, 2.09, 2.1, 2.06, 2.12, 2.08, 2, 1.93, 2.02, 2.55, 1.54, 1.64, - 1.51, 1.55, 2.82, 2.92, 2.55, 2.37, 1.85, 1.6, 1.72, 1.74, 1.79, 1.9, 1.94, - 2, 2.04, 2.08, 2.12, 2.13, 2.16, 2.18, 2.18, 2.2, 2.2, 2.41, 2.39, 2.38, 2.4, - 2.42, 2.41, 2.43, 2.45, 2.43, 2.45, 2.43, 2.4, 2.44, 2.4, 2.42, 2.43, 2.45, - 2.45, 2.45, 2.46, 2.45, 2.45, 2.43, 2.51, 2.48, 2.48, 2.53, 2.46, 2.49, 2.5, - 2.5, 2.5, 2.52, 2.52, 2.54, 2.5, 2.48, 2.5, 2.55, 2.5, 2.48, 2.5, 2.5, 2.52, - 2.52, 2.48, 2.5, 2.5, 2.52, 2.46, 2.53, 9] + error parameter vector: &{{observation_from_jcb}}_oberr [{{ get_satellite_variable(observation_from_jcb, "error") }}] # Step 4: Remove Observations from the Edge of the Scan - filter: Domain Check @@ -163,7 +116,6 @@ name: MetaData/sensorScanPosition is_in: 5-56 action: - name: reject name: reduce obs space # Step 5: Data Thinning @@ -175,10 +127,7 @@ use_reduced_horizontal_grid: true distance_norm: geodesic priority_variable: MetaData/fractionOfClearPixelsInFOV - # round_horizontal_bin_count_to_nearest: true - # partition_longitude_bins_using_mesh: true action: - name: reject name: reduce obs space # Observation Post Filters (QC) @@ -198,7 +147,7 @@ name: MetaData/solarZenithAngle maxvalue: 88.9999 - variable: - name: water_area_fraction@GeoVaLs + name: GeoVaLs/water_area_fraction minvalue: 1.0e-12 action: name: reject @@ -211,7 +160,7 @@ action: name: inflate error inflation variable: - name: ObsErrorFactorWavenumIR@ObsFunction + name: ObsFunction/ObsErrorFactorWavenumIR channels: *{{observation_from_jcb}}_simulated_channels options: channels: *{{observation_from_jcb}}_simulated_channels @@ -264,71 +213,8 @@ channels: *{{observation_from_jcb}}_simulated_channels options: channels: *{{observation_from_jcb}}_simulated_channels - use_flag: [1, -1, -1, -1, 1, -1, -1, -1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, - 1, 1, -1, 1, 1, 1, -1, -1, 1, 1, -1, 1, -1, 1, -1, 1, -1, -1, -1, -1, -1, - -1, -1, -1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, -1, - 1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, -1, -1, - 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, 1, 1, 1, - 1, 1, 1, -1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, 1, 1, -1, -1, -1, - 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, - 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, -1, 1, 1, 1, -1, 1, 1, -1, 1, -1, - 1, -1, -1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, -1, -1, -1, 1, 1, 1, -1, - -1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, 1, -1, 1, 1, -1, 1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, - -1, -1, 1, -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, 1, - -1, 1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, -1, -1, -1, -1, -1, 1, -1, 1, -1, - 1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, - -1, -1, 1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, - -1, -1, -1, -1, -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, 1, - -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1] - use_flag_clddet: [1, -1, -1, -1, 1, -1, -1, -1, 1, -1, 1, -1, 1, -1, 1, 1, - 1, -1, 1, 1, -1, 1, 1, 1, -1, -1, 1, 1, -1, 1, -1, 1, -1, 1, -1, -1, -1, - -1, -1, -1, -1, -1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, - 1, 1, -1, 1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, - -1, -1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, 1, - 1, 1, 1, 1, 1, -1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, 1, 1, -1, - -1, -1, 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, 1, - 1, 1, 1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, -1, 1, 1, 1, -1, 1, 1, - -1, 1, -1, 1, -1, -1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, -1, -1, -1, 1, - 1, 1, -1, -1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, 1, -1, 1, 1, -1, 1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, 1, -1, 1, -1, 1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, - -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1] + use_flag: &{{observation_from_jcb}}_active_channels [{{ get_satellite_variable(observation_from_jcb, "active") }}] + use_flag_clddet: [{{ get_satellite_variable(observation_from_jcb, "icld_det") }}] obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] error parameter vector: *{{observation_from_jcb}}_oberr maxvalue: 1.0e-12 @@ -341,42 +227,11 @@ - name: brightnessTemperature channels: *{{observation_from_jcb}}_simulated_channels test variables: - - name: NearSSTRetCheckIR@ObsFunction + - name: ObsFunction/NearSSTRetCheckIR channels: *{{observation_from_jcb}}_simulated_channels options: channels: *{{observation_from_jcb}}_simulated_channels - use_flag: [1, -1, -1, -1, 1, -1, -1, -1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, - 1, 1, -1, 1, 1, 1, -1, -1, 1, 1, -1, 1, -1, 1, -1, 1, -1, -1, -1, -1, -1, - -1, -1, -1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, -1, - 1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, -1, -1, - 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, 1, 1, 1, - 1, 1, 1, -1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, 1, 1, -1, -1, -1, - 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, - 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, -1, 1, 1, 1, -1, 1, 1, -1, 1, -1, - 1, -1, -1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, -1, -1, -1, 1, 1, 1, -1, - -1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, 1, -1, 1, 1, -1, 1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, - -1, -1, 1, -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, 1, - -1, 1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, -1, -1, -1, -1, -1, 1, -1, 1, -1, - 1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, - -1, -1, 1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, - -1, -1, -1, -1, -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, 1, - -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1] + use_flag: *{{observation_from_jcb}}_active_channels obserr_demisf: [0.01, 0.02, 0.03, 0.02, 0.03] obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] maxvalue: 1.0e-12 @@ -418,48 +273,7 @@ channels: *{{observation_from_jcb}}_simulated_channels options: channels: *{{observation_from_jcb}}_simulated_channels - obserr_bound_max: [3.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 4.0, 2.0, 2.0, 2.0, - 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 4.0, 4.0, 3.5, - 2.5, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 3.5, 2.0, 2.0, 2.0, 2.0, 2.0, - 2.0, 3.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - 2.0, 2.0, 2.0, 2.5, 2.0, 2.5, 2.5, 3.0, 2.5, 2.5, 2.5, 2.5, 3.5, 2.5, 2.5, - 3.0, 3.5, 3.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.5, 4.5, 4.5, 4.5, 4.5, 4.0, - 4.5, 4.0, 4.0, 4.5, 2.5, 3.0, 2.5, 3.0, 2.5, 3.0, 2.0, 2.5, 2.5, 3.0, 3.0, - 2.5, 3.0, 3.0, 3.0, 2.5, 2.5, 4.0, 4.5, 4.5, 5.0, 4.0, 4.0, 5.0, 5.0, 5.0, - 5.0, 5.5, 5.5, 4.0, 5.0, 4.0, 4.5, 5.5, 5.5, 6.0, 4.5, 4.5, 4.0, 5.0, 5.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 5.5, 4.5, 6.0, 5.0, 5.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 5.0, 6.0, 6.0, 6.0, 4.0, 6.0, 6.0, 6.0, 6.0, 4.5, 6.0, 6.0, 4.5, - 6.0, 6.0, 6.0, 6.0, 6.0, 5.0, 6.0, 6.0, 6.0, 5.0, 6.0, 6.0, 5.0, 6.0, 5.0, - 6.0, 6.0, 6.0, 5.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, - 6.0, 6.0, 6.0, 6.0, 6.0] + obserr_bound_max: [{{ get_satellite_variable(observation_from_jcb, "ermax") }}] error parameter vector: *{{observation_from_jcb}}_oberr action: name: reject @@ -474,38 +288,7 @@ channels: *{{observation_from_jcb}}_simulated_channels options: channels: *{{observation_from_jcb}}_simulated_channels - use_flag: [1, -1, -1, -1, 1, -1, -1, -1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, - 1, 1, -1, 1, 1, 1, -1, -1, 1, 1, -1, 1, -1, 1, -1, 1, -1, -1, -1, -1, -1, - -1, -1, -1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, -1, - 1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, -1, -1, - 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, 1, 1, 1, - 1, 1, 1, -1, 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, 1, -1, 1, 1, -1, -1, -1, - 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, - 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, 1, -1, -1, 1, 1, 1, -1, 1, 1, -1, 1, -1, - 1, -1, -1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, -1, -1, -1, 1, 1, 1, -1, - -1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, 1, -1, 1, 1, -1, 1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, - -1, -1, 1, -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, 1, - -1, 1, 1, -1, 1, 1, 1, 1, 1, -1, -1, 1, -1, -1, -1, -1, -1, 1, -1, 1, -1, - 1, -1, -1, -1, 1, 1, 1, 1, 1, 1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, 1, - -1, -1, 1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, - -1, -1, -1, -1, -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, 1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, 1, - -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, -1, -1, -1] + use_flag: *{{observation_from_jcb}}_active_channels minvalue: 1.0e-12 action: name: reject From e08b13abc0fe5063e7bfbacaa1598f7d0ee96b59 Mon Sep 17 00:00:00 2001 From: emilyhcliu <36091766+emilyhcliu@users.noreply.github.com> Date: Mon, 7 Apr 2025 10:31:51 -0400 Subject: [PATCH 104/199] Update SSMIS yaml to match GSI results (#100) Update SSMIS filters to match GSI results. For details regarding the end-to-end evaluation between GSI and JEDI, please GDASApp[ Issue #1563](https://github.com/NOAA-EMC/GDASApp/issues/1563) --------- Co-authored-by: Dan Holdaway <27729500+danholdaway@users.noreply.github.com> --- .../observations/atmosphere/ssmis_f17.yaml.j2 | 833 +++++++++++++++-- .../observations/atmosphere/ssmis_f18.yaml.j2 | 835 ++++++++++++++++-- 2 files changed, 1525 insertions(+), 143 deletions(-) diff --git a/parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 index ef47275c6..34f762711 100644 --- a/parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 @@ -12,8 +12,10 @@ engine: type: H5File obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" - simulated variables: [brightness_temperature] - channels: {{ get_satellite_variable(observation_from_jcb, "simulated") }} + io pool: + max pool size: 1 + simulated variables: [brightnessTemperature] + channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} # Observation Operator # -------------------- @@ -24,6 +26,8 @@ Sensor_ID: &{{observation_from_jcb}}_sensor_id ssmis_f17 EndianType: little_endian CoefficientPath: "{{crtm_coefficient_path}}" + linear obs operator: + Absorbers: [H2O, O3] # Observation Bias Correction (VarBC) # ----------------------------------- @@ -33,7 +37,7 @@ variational bc: predictors: - name: constant - - name: cloud_liquid_water + - name: cloudWaterContent sensor: SSMIS ch19h: 12 ch19v: 13 @@ -42,8 +46,8 @@ ch37v: 16 ch91v: 17 ch91h: 18 - - name: cosine_of_latitude_times_orbit_node - - name: sine_of_latitude + - name: cosineOfLatitudeTimesOrbitNode + - name: sineOfLatitude - name: lapseRate order: 2 tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" @@ -51,122 +55,809 @@ tlapse: *{{observation_from_jcb}}_tlapse - name: emissivityJacobian - name: sensorScanAngle - var_name: scan_position + var_name: sensorScanPosition order: 4 - name: sensorScanAngle - var_name: scan_position + var_name: sensorScanPosition order: 3 - name: sensorScanAngle - var_name: scan_position + var_name: sensorScanPosition order: 2 - name: sensorScanAngle - var_name: scan_position + var_name: sensorScanPosition + covariance: + minimal required obs number: 20 + variance range: [1.0e-6, 10.0] + step size: 1.0e-4 + largest analysis variance: 10000.0 + prior: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiascovin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovin_suffix}}" + inflation: + ratio: 1.1 + ratio for small dataset: 2.0 + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" - # Observation Prior Filters (QC) + # Observation Pre Filters (QC) # ------------------------------ - obs prior filters: + obs pre filters: + # Step 1: Remove missing data from the obs space + - filter: RejectList + where: + - variable: + name: ObsValue/brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + value: is_not_valid + action: + name: reduce obs space + + # Step 2: Create thinning scores based on surface types + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/thinningScore + type: int + value: 0 + + - filter: Variable Assignment + where: + - variable: + name: MetaData/earthSurfaceType + is_in: 5, 7, 10 # ocean/water + assignments: + - name: DerivedMetaData/thinningScore + type: int + value: 100 + + - filter: Variable Assignment + where: + - variable: + name: MetaData/earthSurfaceType + is_in: 0 , 12 # land + assignments: + - name: DerivedMetaData/thinningScore + type: int + value: 85 + + - filter: Variable Assignment + where: + - variable: + name: MetaData/earthSurfaceType + is_in: 3, 4, 9 # Ice/Sea Ice/Frozen Soil + assignments: + - name: DerivedMetaData/thinningScore + type: int + value: 90 + + - filter: Variable Assignment + where: + - variable: + name: MetaData/earthSurfaceType + is_in: 8, 11 # Snow + assignments: + - name: DerivedMetaData/thinningScore + type: int + value: 85 + + # Step 3: Data Thinning + - filter: Gaussian Thinning + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + horizontal_mesh: 145 + use_reduced_horizontal_grid: true + distance_norm: geodesic + priority_variable: DerivedMetaData/thinningScore + action: + name: reduce obs space + + # Step 4: Initial Observation Error Assignment - filter: Perform Action filter variables: - - name: brightness_temperature - channels: 1-24 + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels action: name: assign error - error parameter vector: [1.5, 0.5, 0.5, 0.5, 0.5, 1, 1, 3, 3, 3, 3, 2.4, 1.27, - 1.44, 3, 1.34, 1.74, 3.75, 3, 3, 2, 6.4, 1, 1] + error parameter vector: &{{observation_from_jcb}}_oberr [{{ get_satellite_variable(observation_from_jcb, "error") }}] # Observation Post Filters (QC) - # ----------------------------- + # ------------------------------ obs post filters: - #step1: Gross check (setuprad) + # Step 5: Gross check over water-dominant area - filter: Background Check filter variables: - - name: brightness_temperature - channels: 1-24 - threshold: 1.5 - action: - name: reject - #step1: Gross check(qcmod) - - filter: Background Check - filter variables: - - name: brightness_temperature - channels: 1-24 + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels absolute threshold: 3.5 remove bias correction: true + where: + - variable: + name: GeoVaLs/water_area_fraction + minvalue: 0.99 action: name: reject - - filter: Difference Check + # Step 6: Reject all channels if surface height is greater than 2km + - filter: RejectList filter variables: - - name: brightness_temperature - channels: 1-2,12-16 - reference: brightness_temperature_2@ObsValue - value: brightness_temperature_2@HofX - minvalue: -1.5 - maxvalue: 1.5 + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels where: - variable: - name: water_area_fraction@GeoVaLs - maxvalue: 0.99 - #QC_terrain: If seviri and terrain height > 2km. do not use - - filter: Domain Check - filter variables: - - name: brightness_temperature - channels: 1-24 - where: + name: GeoVaLs/geopotential_height_at_surface + minvalue: 2000.0 + min_exclusive: true - variable: - name: height_above_mean_sea_level@MetaData - maxvalue: 2000.0 - #do not use over mixed surface - - filter: BlackList + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + max_exclusive: true + + # Step 7: Reject data over mixed surface type + - filter: RejectList filter variables: - - name: brightness_temperature + - name: brightnessTemperature channels: 1-3,8-18 where: - variable: - name: land_area_fraction@GeoVaLs + name: GeoVaLs/land_area_fraction maxvalue: 0.99 + max_exclusive: true - variable: - name: water_area_fraction@GeoVaLs + name: GeoVaLs/water_area_fraction maxvalue: 0.99 + max_exclusive: true - variable: - name: ice_area_fraction@GeoVaLs + name: GeoVaLs/ice_area_fraction maxvalue: 0.99 + max_exclusive: true - variable: - name: surface_snow_area_fraction@GeoVaLs + name: GeoVaLs/surface_snow_area_fraction maxvalue: 0.99 - #step4: Generate q.c. bounds and modified variances - - filter: BlackList + max_exclusive: true + + # Step 8: Channel 2 O-F check over non-water dominant area + - filter: Difference Check filter variables: - - name: brightness_temperature - channels: 1-24 + - name: brightnessTemperature + channels: 1-2, 12-16 + reference: ObsValue/brightnessTemperature_2 + value: HofX/brightnessTemperature_2 + minvalue: -1.5 + maxvalue: 1.5 + where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + max_exclusive: true + + # Step 9: Gross check over non-water dominant area + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + absolute threshold: 3.5 + remove bias correction: true + where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + max_exclusive: true action: - name: inflate error - inflation variable: - # Surface Jacobian check - name: ObsErrorFactorSurfJacobianRad@ObsFunction - channels: 1-24 + name: reject + + # Step 10: Scattering check for channels 9-11 using channels 8 and 17 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/SSMISScatteringIndex9 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: HofX/brightnessTemperature_17 + - name: ObsBiasData/brightnessTemperature_17 + - name: HofX/brightnessTemperature_8 + - name: ObsBiasData/brightnessTemperature_8 + coefs: [-0.485934, 0.485934, 0.473806, -0.473806] + intercept: 271.252327 + + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/SSMISScatteringIndex10 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: HofX/brightnessTemperature_17 + - name: ObsBiasData/brightnessTemperature_17 + - name: HofX/brightnessTemperature_8 + - name: ObsBiasData/brightnessTemperature_8 + coefs: [-0.413688, 0.413688, 0.361549, -0.361549] + intercept: 272.280341 + + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/SSMISScatteringIndex11 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: HofX/brightnessTemperature_17 + - name: ObsBiasData/brightnessTemperature_17 + - name: HofX/brightnessTemperature_8 + - name: ObsBiasData/brightnessTemperature_8 + coefs: [-0.400882, 0.400882, 0.270510, -0.270510] + intercept: 278.824902 + + - filter: Difference Check + filter variables: + - name: brightnessTemperature + channels: 9 + reference: ObsValue/brightnessTemperature_9 + value: DerivedMetaData/SSMISScatteringIndex9 + maxvalue: 2 + + - filter: Difference Check + filter variables: + - name: brightnessTemperature + channels: 10 + reference: ObsValue/brightnessTemperature_10 + value: DerivedMetaData/SSMISScatteringIndex10 + maxvalue: 2 + + - filter: Difference Check + filter variables: + - name: brightnessTemperature + channels: 11 + reference: ObsValue/brightnessTemperature_11 + value: DerivedMetaData/SSMISScatteringIndex11 + maxvalue: 2 + + # Step 11: Error inflation based on surface jacobian + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ObsErrorFactorSurfJacobian + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsFunction/ObsErrorFactorSurfJacobianRad + channels: *{{observation_from_jcb}}_simulated_channels options: - channels: 1-24 sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels obserr_demisf: [0.010, 0.010, 0.010, 0.010, 0.010] obserr_dtempf: [0.500, 0.500, 0.500, 0.500, 0.500] - # Useflag Check + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: DerivedMetaData/ObsErrorFactorSurfJacobian + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 12: Final gross check + # Channel 1 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor1 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_1 + coefs: [ 2.25 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_1 + threshold: DerivedMetaData/errorInflationFactor1 + absolute threshold: 6.0 + action: + name: reject + + # Channel 2 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor2 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_2 + coefs: [ 0.75 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_2 + threshold: DerivedMetaData/errorInflationFactor2 + absolute threshold: 6.0 + action: + name: reject + + # Channel 3 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor3 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_3 + coefs: [ 0.75 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_3 + threshold: DerivedMetaData/errorInflationFactor3 + absolute threshold: 6.0 + action: + name: reject + + # Channel 4 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor4 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_4 + coefs: [ 0.75 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_4 + threshold: DerivedMetaData/errorInflationFactor4 + absolute threshold: 6.0 + action: + name: reject + + # Channel 5 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor5 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_5 + coefs: [ 0.75 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_5 + threshold: DerivedMetaData/errorInflationFactor5 + absolute threshold: 6.0 + action: + name: reject + + # Channel 6 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor6 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_6 + coefs: [ 1.50 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_6 + threshold: DerivedMetaData/errorInflationFactor6 + absolute threshold: 6.0 + action: + name: reject + + # Channel 7 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor7 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_7 + coefs: [ 1.50 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_7 + threshold: DerivedMetaData/errorInflationFactor7 + absolute threshold: 6.0 + action: + name: reject + + # Channel 8 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor8 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_8 + coefs: [ 4.50 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_8 + threshold: DerivedMetaData/errorInflationFactor8 + absolute threshold: 6.0 + action: + name: reject + + # Channel 9 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor9 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_9 + coefs: [ 4.50 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_9 + threshold: DerivedMetaData/errorInflationFactor9 + absolute threshold: 6.0 + action: + name: reject + + # Channel 10 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor10 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_10 + coefs: [ 4.50 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_10 + threshold: DerivedMetaData/errorInflationFactor10 + absolute threshold: 6.0 + action: + name: reject + + # Channel 11 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor11 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_11 + coefs: [ 4.50 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_11 + threshold: DerivedMetaData/errorInflationFactor11 + absolute threshold: 6.0 + action: + name: reject + + # Channel 12 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor12 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_12 + coefs: [ 3.60 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_12 + threshold: DerivedMetaData/errorInflationFactor12 + absolute threshold: 6.0 + action: + name: reject + + # Channel 13 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor13 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_13 + coefs: [ 1.905 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_13 + threshold: DerivedMetaData/errorInflationFactor13 + absolute threshold: 6.0 + action: + name: reject + + # Channel 14 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor14 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_14 + coefs: [ 2.16 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_14 + threshold: DerivedMetaData/errorInflationFactor14 + absolute threshold: 6.0 + action: + name: reject + + # Channel 15 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor15 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_15 + coefs: [ 4.50 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_15 + threshold: DerivedMetaData/errorInflationFactor15 + absolute threshold: 6.0 + action: + name: reject + + # Channel 16 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor16 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_16 + coefs: [ 2.01 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_16 + threshold: DerivedMetaData/errorInflationFactor16 + absolute threshold: 6.0 + action: + name: reject + + # Channel 17 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor17 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_17 + coefs: [ 2.61 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_17 + threshold: DerivedMetaData/errorInflationFactor17 + absolute threshold: 6.0 + action: + name: reject + + # Channel 18 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor18 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_18 + coefs: [ 5.625 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_18 + threshold: DerivedMetaData/errorInflationFactor18 + absolute threshold: 6.0 + action: + name: reject + + # Channel 19 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor19 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_19 + coefs: [ 4.50 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_19 + threshold: DerivedMetaData/errorInflationFactor19 + absolute threshold: 6.0 + action: + name: reject + + # Channel 20 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor20 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_20 + coefs: [ 4.50 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_20 + threshold: DerivedMetaData/errorInflationFactor20 + absolute threshold: 6.0 + action: + name: reject + + # Channel 21 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor21 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_21 + coefs: [ 3.00 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_21 + threshold: DerivedMetaData/errorInflationFactor21 + absolute threshold: 6.0 + action: + name: reject + + # Channel 22 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor22 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_22 + coefs: [ 9.60 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_22 + threshold: DerivedMetaData/errorInflationFactor22 + absolute threshold: 6.0 + action: + name: reject + + # Channel 23 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor23 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_23 + coefs: [ 1.50 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_23 + threshold: DerivedMetaData/errorInflationFactor23 + absolute threshold: 6.0 + action: + name: reject + + # Channel 24 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor24 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_24 + coefs: [ 1.50 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_24 + threshold: DerivedMetaData/errorInflationFactor24 + absolute threshold: 6.0 + action: + name: reject + + # Step 13: Useflag Check - filter: Bounds Check filter variables: - - name: brightness_temperature - channels: 1-24 + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels test variables: - - name: ChannelUseflagCheckRad@ObsFunction - channels: 1-24 + - name: ObsFunction/ChannelUseflagCheckRad + channels: *{{observation_from_jcb}}_simulated_channels options: - channels: 1-24 - use_flag: [1, -1, -1, -1, 1, 1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, 1] + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: &{{observation_from_jcb}}_active_channels [{{ get_satellite_variable(observation_from_jcb, "active") }}] minvalue: 1.0e-12 action: name: reject - # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: @@ -174,4 +865,4 @@ # Passed benchmark for UFO testing # -------------------------------- - passedBenchmark: 0 + passedBenchmark: 126336 diff --git a/parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 index feffb1e84..cc74f9aeb 100644 --- a/parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 @@ -12,8 +12,10 @@ engine: type: H5File obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" - simulated variables: [brightness_temperature] - channels: {{ get_satellite_variable(observation_from_jcb, "simulated") }} + io pool: + max pool size: 1 + simulated variables: [brightnessTemperature] + channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} # Observation Operator # -------------------- @@ -21,9 +23,11 @@ name: CRTM Absorbers: [H2O, O3, CO2] obs options: - Sensor_ID: &{{observation_from_jcb}}_sensor_id ssmis_f17 + Sensor_ID: &{{observation_from_jcb}}_sensor_id ssmis_f18 EndianType: little_endian CoefficientPath: "{{crtm_coefficient_path}}" + linear obs operator: + Absorbers: [H2O, O3] # Observation Bias Correction (VarBC) # ----------------------------------- @@ -33,7 +37,7 @@ variational bc: predictors: - name: constant - - name: cloud_liquid_water + - name: cloudWaterContent sensor: SSMIS ch19h: 12 ch19v: 13 @@ -42,8 +46,8 @@ ch37v: 16 ch91v: 17 ch91h: 18 - - name: cosine_of_latitude_times_orbit_node - - name: sine_of_latitude + - name: cosineOfLatitudeTimesOrbitNode + - name: sineOfLatitude - name: lapseRate order: 2 tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" @@ -51,122 +55,809 @@ tlapse: *{{observation_from_jcb}}_tlapse - name: emissivityJacobian - name: sensorScanAngle - var_name: scan_position + var_name: sensorScanPosition order: 4 - name: sensorScanAngle - var_name: scan_position + var_name: sensorScanPosition order: 3 - name: sensorScanAngle - var_name: scan_position + var_name: sensorScanPosition order: 2 - name: sensorScanAngle - var_name: scan_position + var_name: sensorScanPosition + covariance: + minimal required obs number: 20 + variance range: [1.0e-6, 10.0] + step size: 1.0e-4 + largest analysis variance: 10000.0 + prior: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiascovin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovin_suffix}}" + inflation: + ratio: 1.1 + ratio for small dataset: 2.0 + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" - # Observation Prior Filters (QC) + # Observation Pre Filters (QC) # ------------------------------ - obs prior filters: + obs pre filters: + # Step 1: Remove missing data from the obs space + - filter: RejectList + where: + - variable: + name: ObsValue/brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + value: is_not_valid + action: + name: reduce obs space + + # Step 2: Create thinning scores based on surface types + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/thinningScore + type: int + value: 0 + + - filter: Variable Assignment + where: + - variable: + name: MetaData/earthSurfaceType + is_in: 5, 7, 10 # ocean/water + assignments: + - name: DerivedMetaData/thinningScore + type: int + value: 100 + + - filter: Variable Assignment + where: + - variable: + name: MetaData/earthSurfaceType + is_in: 0 , 12 # land + assignments: + - name: DerivedMetaData/thinningScore + type: int + value: 85 + + - filter: Variable Assignment + where: + - variable: + name: MetaData/earthSurfaceType + is_in: 3, 4, 9 # Ice/Sea Ice/Frozen Soil + assignments: + - name: DerivedMetaData/thinningScore + type: int + value: 90 + + - filter: Variable Assignment + where: + - variable: + name: MetaData/earthSurfaceType + is_in: 8, 11 # Snow + assignments: + - name: DerivedMetaData/thinningScore + type: int + value: 85 + + # Step 3: Data Thinning + - filter: Gaussian Thinning + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + horizontal_mesh: 145 + use_reduced_horizontal_grid: true + distance_norm: geodesic + priority_variable: DerivedMetaData/thinningScore + action: + name: reduce obs space + + # Step 4: Initial Observation Error Assignment - filter: Perform Action filter variables: - - name: brightness_temperature - channels: 1-24 + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels action: name: assign error - error parameter vector: [1.5, 0.5, 0.5, 0.5, 0.5, 1, 1, 3, 3, 3, 3, 2.4, 1.27, - 1.44, 3, 1.34, 1.74, 3.75, 3, 3, 2, 6.4, 1, 1] + error parameter vector: &{{observation_from_jcb}}_oberr [{{ get_satellite_variable(observation_from_jcb, "error") }}] # Observation Post Filters (QC) - # ----------------------------- + # ------------------------------ obs post filters: - #step1: Gross check (setuprad) + # Step 5: Gross check over water-dominant area - filter: Background Check filter variables: - - name: brightness_temperature - channels: 1-24 - threshold: 1.5 - action: - name: reject - #step1: Gross check(qcmod) - - filter: Background Check - filter variables: - - name: brightness_temperature - channels: 1-24 + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels absolute threshold: 3.5 remove bias correction: true + where: + - variable: + name: GeoVaLs/water_area_fraction + minvalue: 0.99 action: name: reject - - filter: Difference Check + # Step 6: Reject all channels if surface height is greater than 2km + - filter: RejectList filter variables: - - name: brightness_temperature - channels: 1-2,12-16 - reference: brightness_temperature_2@ObsValue - value: brightness_temperature_2@HofX - minvalue: -1.5 - maxvalue: 1.5 + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels where: - variable: - name: water_area_fraction@GeoVaLs - maxvalue: 0.99 - #QC_terrain: If seviri and terrain height > 2km. do not use - - filter: Domain Check - filter variables: - - name: brightness_temperature - channels: 1-24 - where: + name: GeoVaLs/geopotential_height_at_surface + minvalue: 2000.0 + min_exclusive: true - variable: - name: height_above_mean_sea_level@MetaData - maxvalue: 2000.0 - #do not use over mixed surface - - filter: BlackList + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + max_exclusive: true + + # Step 7: Reject data over mixed surface type + - filter: RejectList filter variables: - - name: brightness_temperature + - name: brightnessTemperature channels: 1-3,8-18 where: - variable: - name: land_area_fraction@GeoVaLs + name: GeoVaLs/land_area_fraction maxvalue: 0.99 + max_exclusive: true - variable: - name: water_area_fraction@GeoVaLs + name: GeoVaLs/water_area_fraction maxvalue: 0.99 + max_exclusive: true - variable: - name: ice_area_fraction@GeoVaLs + name: GeoVaLs/ice_area_fraction maxvalue: 0.99 + max_exclusive: true - variable: - name: surface_snow_area_fraction@GeoVaLs + name: GeoVaLs/surface_snow_area_fraction maxvalue: 0.99 - #step4: Generate q.c. bounds and modified variances - - filter: BlackList + max_exclusive: true + + # Step 8: Channel 2 O-F check over non-water dominant area + - filter: Difference Check filter variables: - - name: brightness_temperature - channels: 1-24 + - name: brightnessTemperature + channels: 1-2, 12-16 + reference: ObsValue/brightnessTemperature_2 + value: HofX/brightnessTemperature_2 + minvalue: -1.5 + maxvalue: 1.5 + where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + max_exclusive: true + + # Step 9: Gross check over non-water dominant area + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + absolute threshold: 3.5 + remove bias correction: true + where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + max_exclusive: true action: - name: inflate error - inflation variable: - # Surface Jacobian check - name: ObsErrorFactorSurfJacobianRad@ObsFunction - channels: 1-24 + name: reject + + # Step 10: Scattering check for channels 9-11 using channels 8 and 17 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/SSMISScatteringIndex9 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: HofX/brightnessTemperature_17 + - name: ObsBiasData/brightnessTemperature_17 + - name: HofX/brightnessTemperature_8 + - name: ObsBiasData/brightnessTemperature_8 + coefs: [-0.485934, 0.485934, 0.473806, -0.473806] + intercept: 271.252327 + + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/SSMISScatteringIndex10 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: HofX/brightnessTemperature_17 + - name: ObsBiasData/brightnessTemperature_17 + - name: HofX/brightnessTemperature_8 + - name: ObsBiasData/brightnessTemperature_8 + coefs: [-0.413688, 0.413688, 0.361549, -0.361549] + intercept: 272.280341 + + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/SSMISScatteringIndex11 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: HofX/brightnessTemperature_17 + - name: ObsBiasData/brightnessTemperature_17 + - name: HofX/brightnessTemperature_8 + - name: ObsBiasData/brightnessTemperature_8 + coefs: [-0.400882, 0.400882, 0.270510, -0.270510] + intercept: 278.824902 + + - filter: Difference Check + filter variables: + - name: brightnessTemperature + channels: 9 + reference: ObsValue/brightnessTemperature_9 + value: DerivedMetaData/SSMISScatteringIndex9 + maxvalue: 2 + + - filter: Difference Check + filter variables: + - name: brightnessTemperature + channels: 10 + reference: ObsValue/brightnessTemperature_10 + value: DerivedMetaData/SSMISScatteringIndex10 + maxvalue: 2 + + - filter: Difference Check + filter variables: + - name: brightnessTemperature + channels: 11 + reference: ObsValue/brightnessTemperature_11 + value: DerivedMetaData/SSMISScatteringIndex11 + maxvalue: 2 + + # Step 11: Error inflation based on surface jacobian + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ObsErrorFactorSurfJacobian + channels: *{{observation_from_jcb}}_simulated_channels + type: float + function: + name: ObsFunction/ObsErrorFactorSurfJacobianRad + channels: *{{observation_from_jcb}}_simulated_channels options: - channels: 1-24 sensor: *{{observation_from_jcb}}_sensor_id + channels: *{{observation_from_jcb}}_simulated_channels obserr_demisf: [0.010, 0.010, 0.010, 0.010, 0.010] obserr_dtempf: [0.500, 0.500, 0.500, 0.500, 0.500] - # Useflag Check + + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: DerivedMetaData/ObsErrorFactorSurfJacobian + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 12: Final gross check + # Channel 1 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor1 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_1 + coefs: [ 2.25 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_1 + threshold: DerivedMetaData/errorInflationFactor1 + absolute threshold: 6.0 + action: + name: reject + + # Channel 2 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor2 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_2 + coefs: [ 0.75 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_2 + threshold: DerivedMetaData/errorInflationFactor2 + absolute threshold: 6.0 + action: + name: reject + + # Channel 3 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor3 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_3 + coefs: [ 0.75 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_3 + threshold: DerivedMetaData/errorInflationFactor3 + absolute threshold: 6.0 + action: + name: reject + + # Channel 4 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor4 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_4 + coefs: [ 0.75 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_4 + threshold: DerivedMetaData/errorInflationFactor4 + absolute threshold: 6.0 + action: + name: reject + + # Channel 5 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor5 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_5 + coefs: [ 0.75 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_5 + threshold: DerivedMetaData/errorInflationFactor5 + absolute threshold: 6.0 + action: + name: reject + + # Channel 6 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor6 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_6 + coefs: [ 1.50 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_6 + threshold: DerivedMetaData/errorInflationFactor6 + absolute threshold: 6.0 + action: + name: reject + + # Channel 7 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor7 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_7 + coefs: [ 1.50 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_7 + threshold: DerivedMetaData/errorInflationFactor7 + absolute threshold: 6.0 + action: + name: reject + + # Channel 8 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor8 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_8 + coefs: [ 4.50 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_8 + threshold: DerivedMetaData/errorInflationFactor8 + absolute threshold: 6.0 + action: + name: reject + + # Channel 9 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor9 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_9 + coefs: [ 4.50 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_9 + threshold: DerivedMetaData/errorInflationFactor9 + absolute threshold: 6.0 + action: + name: reject + + # Channel 10 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor10 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_10 + coefs: [ 4.50 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_10 + threshold: DerivedMetaData/errorInflationFactor10 + absolute threshold: 6.0 + action: + name: reject + + # Channel 11 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor11 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_11 + coefs: [ 4.50 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_11 + threshold: DerivedMetaData/errorInflationFactor11 + absolute threshold: 6.0 + action: + name: reject + + # Channel 12 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor12 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_12 + coefs: [ 3.60 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_12 + threshold: DerivedMetaData/errorInflationFactor12 + absolute threshold: 6.0 + action: + name: reject + + # Channel 13 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor13 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_13 + coefs: [ 1.905 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_13 + threshold: DerivedMetaData/errorInflationFactor13 + absolute threshold: 6.0 + action: + name: reject + + # Channel 14 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor14 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_14 + coefs: [ 2.16 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_14 + threshold: DerivedMetaData/errorInflationFactor14 + absolute threshold: 6.0 + action: + name: reject + + # Channel 15 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor15 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_15 + coefs: [ 4.50 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_15 + threshold: DerivedMetaData/errorInflationFactor15 + absolute threshold: 6.0 + action: + name: reject + + # Channel 16 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor16 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_16 + coefs: [ 2.01 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_16 + threshold: DerivedMetaData/errorInflationFactor16 + absolute threshold: 6.0 + action: + name: reject + + # Channel 17 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor17 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_17 + coefs: [ 2.61 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_17 + threshold: DerivedMetaData/errorInflationFactor17 + absolute threshold: 6.0 + action: + name: reject + + # Channel 18 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor18 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_18 + coefs: [ 5.625 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_18 + threshold: DerivedMetaData/errorInflationFactor18 + absolute threshold: 6.0 + action: + name: reject + + # Channel 19 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor19 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_19 + coefs: [ 4.50 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_19 + threshold: DerivedMetaData/errorInflationFactor19 + absolute threshold: 6.0 + action: + name: reject + + # Channel 20 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor20 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_20 + coefs: [ 4.50 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_20 + threshold: DerivedMetaData/errorInflationFactor20 + absolute threshold: 6.0 + action: + name: reject + + # Channel 21 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor21 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_21 + coefs: [ 3.00 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_21 + threshold: DerivedMetaData/errorInflationFactor21 + absolute threshold: 6.0 + action: + name: reject + + # Channel 22 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor22 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_22 + coefs: [ 9.60 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_22 + threshold: DerivedMetaData/errorInflationFactor22 + absolute threshold: 6.0 + action: + name: reject + + # Channel 23 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor23 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_23 + coefs: [ 1.50 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_23 + threshold: DerivedMetaData/errorInflationFactor23 + absolute threshold: 6.0 + action: + name: reject + + # Channel 24 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/errorInflationFactor24 + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsErrorData/brightnessTemperature_24 + coefs: [ 1.50 ] + exponents: [ -1 ] + + - filter: Background Check + filter variables: + - name: brightnessTemperature_24 + threshold: DerivedMetaData/errorInflationFactor24 + absolute threshold: 6.0 + action: + name: reject + + # Step 13: Useflag Check - filter: Bounds Check filter variables: - - name: brightness_temperature - channels: 1-24 + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels test variables: - - name: ChannelUseflagCheckRad@ObsFunction - channels: 1-24 + - name: ObsFunction/ChannelUseflagCheckRad + channels: *{{observation_from_jcb}}_simulated_channels options: - channels: 1-24 - use_flag: [1, -1, -1, -1, 1, 1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - -1, -1, -1, -1, -1, -1, 1] + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: &{{observation_from_jcb}}_active_channels [{{ get_satellite_variable(observation_from_jcb, "active") }}] minvalue: 1.0e-12 action: name: reject - # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- geovals: @@ -174,4 +865,4 @@ # Passed benchmark for UFO testing # -------------------------------- - passedBenchmark: 0 + passedBenchmark: 126336 From 98e7b2dcc798becfdaceaf42560e7cc28367b007 Mon Sep 17 00:00:00 2001 From: jianjunj Date: Mon, 7 Apr 2025 12:25:34 -0400 Subject: [PATCH 105/199] Read ATMS "ermax" in atmosphere_chronicle tables (#102) Get ATMS "obserr_bound_max" from atmosphere_chronicle tables. No impact in results. --- parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 | 3 +-- parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 | 3 +-- parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 | 3 +-- 3 files changed, 3 insertions(+), 6 deletions(-) diff --git a/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 index a46ef0883..d0d8e1d42 100644 --- a/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 @@ -468,8 +468,7 @@ channels: *{{observation_from_jcb}}_simulated_channels threshold: 3 threshold_precip: 2.5 - obserr_bound_max: [4.5, 4.5, 3.0, 3.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, - 1.0, 1.0, 2.0, 4.5, 4.5, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0] + obserr_bound_max: [{{ get_satellite_variable(observation_from_jcb, "ermax") }}] - filter: Background Check filter variables: diff --git a/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 index a631473a3..5e8f12dc4 100644 --- a/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 @@ -468,8 +468,7 @@ channels: *{{observation_from_jcb}}_simulated_channels threshold: 3 threshold_precip: 2.5 - obserr_bound_max: [4.5, 4.5, 3.0, 3.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, - 1.0, 1.0, 2.0, 4.5, 4.5, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0] + obserr_bound_max: [{{ get_satellite_variable(observation_from_jcb, "ermax") }}] - filter: Background Check filter variables: diff --git a/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 index d1e8fb70b..2708a3e0d 100644 --- a/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 @@ -468,8 +468,7 @@ channels: *{{observation_from_jcb}}_simulated_channels threshold: 3 threshold_precip: 2.5 - obserr_bound_max: [4.5, 4.5, 3.0, 3.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, - 1.0, 1.0, 2.0, 4.5, 4.5, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0] + obserr_bound_max: [{{ get_satellite_variable(observation_from_jcb, "ermax") }}] - filter: Background Check filter variables: From 0370563c7867f6e805836f95e6b54031856d0d89 Mon Sep 17 00:00:00 2001 From: jianjunj Date: Fri, 11 Apr 2025 09:42:14 -0400 Subject: [PATCH 106/199] Temporarily reverted cloud_fraction=1.0. (#104) Temporarily turned off the calculations of cloud fraction and particle radii by the Thompson method because of missing geoval "moist_air_density". See the note of it https://github.com/NOAA-EMC/GDASApp/issues/1573#issuecomment-2791277591 . Working on adding the variable in FV-3JEDI. --- parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 | 6 +++--- parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 | 6 +++--- parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 | 6 +++--- parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 | 6 +++--- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 index c1f624250..0fc786614 100644 --- a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 @@ -23,10 +23,10 @@ name: CRTM Absorbers: [H2O, O3, CO2] Clouds: &{{observation_from_jcb}}_clouds [Water, Ice, Rain, Snow, Graupel] -# Cloud_Fraction: 1.0 + Cloud_Fraction: 1.0 # ---methods for cloud fraction and radii in fov: thompson, or none - method for cloud fraction within fov: thompson - method for hydrometeor effective radii within fov: thompson + method for cloud fraction within fov: none + method for hydrometeor effective radii within fov: none Cloud_Seeding: true obs options: Sensor_ID: &{{observation_from_jcb}}_sensor_id amsua_n19 diff --git a/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 index d0d8e1d42..6cbfbac7b 100644 --- a/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 @@ -23,10 +23,10 @@ name: CRTM Absorbers: [H2O, O3, CO2] Clouds: &{{observation_from_jcb}}_clouds [Water, Ice, Rain, Snow, Graupel] -# Cloud_Fraction: 1.0 + Cloud_Fraction: 1.0 # ---methods for cloud fraction and radii in fov: thompson, or none - method for cloud fraction within fov: thompson - method for hydrometeor effective radii within fov: thompson + method for cloud fraction within fov: none + method for hydrometeor effective radii within fov: none Cloud_Seeding: true obs options: Sensor_ID: &{{observation_from_jcb}}_sensor_id atms_n20 diff --git a/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 index 5e8f12dc4..83b627baa 100644 --- a/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 @@ -23,10 +23,10 @@ name: CRTM Absorbers: [H2O, O3, CO2] Clouds: &{{observation_from_jcb}}_clouds [Water, Ice, Rain, Snow, Graupel] -# Cloud_Fraction: 1.0 + Cloud_Fraction: 1.0 # ---methods for cloud fraction and radii in fov: thompson, or none - method for cloud fraction within fov: thompson - method for hydrometeor effective radii within fov: thompson + method for cloud fraction within fov: none + method for hydrometeor effective radii within fov: none Cloud_Seeding: true obs options: Sensor_ID: &{{observation_from_jcb}}_sensor_id atms_n21 diff --git a/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 index 2708a3e0d..f60300209 100644 --- a/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 @@ -23,10 +23,10 @@ name: CRTM Absorbers: [H2O, O3, CO2] Clouds: &{{observation_from_jcb}}_clouds [Water, Ice, Rain, Snow, Graupel] -# Cloud_Fraction: 1.0 + Cloud_Fraction: 1.0 # ---methods for cloud fraction and radii in fov: thompson, or none - method for cloud fraction within fov: thompson - method for hydrometeor effective radii within fov: thompson + method for cloud fraction within fov: none + method for hydrometeor effective radii within fov: none Cloud_Seeding: true obs options: Sensor_ID: &{{observation_from_jcb}}_sensor_id atms_npp From 651f37bea571578c043e5c362d1b384742c59c1a Mon Sep 17 00:00:00 2001 From: JohnSteffen-NOAA <77806417+JohnSteffen-NOAA@users.noreply.github.com> Date: Tue, 15 Apr 2025 14:18:34 -0400 Subject: [PATCH 107/199] Bugfix for marine qc filters (#105) This bugfix PR addresses an ordering issue in the post filters of sst_generic.yaml.j2 and adds "observed variables" that are missing in several of the in situ observation yamls. The missing "observed variables" in bathy, altkob, trkob and trkob_salinity causes the gdas_prepoceanobs task to fail. Using the background check at the end of the post filters in sst_generic.yaml.j2 cause the gdas_marineanlvar task to fail. These bugfix changes were successfully tested on Hera. --- .../observations/marine/insitu_profile_bathy.yaml.j2 | 1 + .../observations/marine/insitu_surface_altkob.yaml.j2 | 1 + .../observations/marine/insitu_surface_trkob.yaml.j2 | 1 + .../marine/insitu_surface_trkob_salinity.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 | 7 +++++-- 5 files changed, 9 insertions(+), 2 deletions(-) diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 index 1811610c7..2f811f062 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 @@ -9,6 +9,7 @@ type: H5File obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" simulated variables: [waterTemperature] + observed variable: [waterTemperature] io pool: max pool size: 1 obs operator: diff --git a/parm/jcb-gdas/observations/marine/insitu_surface_altkob.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_surface_altkob.yaml.j2 index fbe5e1b77..cb8c277c6 100644 --- a/parm/jcb-gdas/observations/marine/insitu_surface_altkob.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_surface_altkob.yaml.j2 @@ -9,6 +9,7 @@ type: H5File obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" simulated variables: [seaSurfaceTemperature] + observed variables: [seaSurfaceTemperature] io pool: max pool size: 1 obs operator: diff --git a/parm/jcb-gdas/observations/marine/insitu_surface_trkob.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_surface_trkob.yaml.j2 index 7a20a4217..be3805708 100644 --- a/parm/jcb-gdas/observations/marine/insitu_surface_trkob.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_surface_trkob.yaml.j2 @@ -9,6 +9,7 @@ type: H5File obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" simulated variables: [seaSurfaceTemperature] + observed variables: [seaSurfaceTemperature] io pool: max pool size: 1 obs operator: diff --git a/parm/jcb-gdas/observations/marine/insitu_surface_trkob_salinity.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_surface_trkob_salinity.yaml.j2 index c7d05e8c2..e162f3bc3 100644 --- a/parm/jcb-gdas/observations/marine/insitu_surface_trkob_salinity.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_surface_trkob_salinity.yaml.j2 @@ -9,6 +9,7 @@ type: H5File obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" simulated variables: [seaSurfaceSalinity] + observed variables: [seaSurfaceSalinity] get values: time interpolation: linear obs operator: diff --git a/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 index 31b5d2d05..b0755aac8 100644 --- a/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 @@ -56,6 +56,8 @@ {% endif %} obs post filters: + - filter: Background Check + absolute threshold: 5.0 - filter: Perform Action # Pamlico Sound & Chesapeake Bay action: name: accept @@ -68,5 +70,6 @@ name: MetaData/longitude minvalue: -76.88 maxvalue: -74.5 - - filter: Background Check - absolute threshold: 5.0 +# TODO: var fails when bkg chk filter is last +# - filter: Background Check +# absolute threshold: 5.0 From 1f04ed99515e436a8dc44673c382edfbc9ffdf9e Mon Sep 17 00:00:00 2001 From: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com> Date: Tue, 22 Apr 2025 14:05:39 -0400 Subject: [PATCH 108/199] updates to allow atmospheric DA with all-sky microwave assimilation (#109) --- parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 | 2 +- .../model/atmosphere/atmosphere_background_ensemble.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 | 2 ++ 3 files changed, 4 insertions(+), 2 deletions(-) diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 index 09a10c955..6c2d847f4 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 @@ -3,6 +3,6 @@ filetype: cube sphere history provider: ufs datetime: "{{ atmosphere_background_time_iso }}" filenames: [ {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_atmf006.nc, {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_sfcf006.nc ] -state variables: [ua,va,t,delp,ps,sphum,ice_wat,liq_wat,o3mr,hgtsfc, +state variables: [ua,va,t,delp,ps,sphum,ice_wat,liq_wat,snmr,rwmr,grle,nccice,nconrd,o3mr,hgtsfc, slmsk,sheleg,tmpsfc,vtype,stype,vfrac,soilt1,soilw1,snwdphMeters, u_srf,v_srf,f10m] diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_background_ensemble.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background_ensemble.yaml.j2 index f3fed38e6..6e7902a72 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_background_ensemble.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background_ensemble.yaml.j2 @@ -4,7 +4,7 @@ members from template: datetime: '{{ atmosphere_background_time_iso }}' filetype: cube sphere history provider: ufs - state variables: [ua,va,t,delz,delp,ps,sphum,ice_wat,liq_wat,o3mr,hgtsfc, + state variables: [ua,va,t,delz,delp,ps,sphum,ice_wat,liq_wat,snmr,rwmr,grle,nccice,nconrd,o3mr,hgtsfc, slmsk,sheleg,tmpsfc,vtype,stype,vfrac,soilt1,soilw1,snwdphMeters, u_srf,v_srf,f10m] datapath: {{ atmosphere_background_ensemble_path }} diff --git a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 index 0fc786614..319eed08d 100644 --- a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 @@ -32,6 +32,8 @@ Sensor_ID: &{{observation_from_jcb}}_sensor_id amsua_n19 EndianType: little_endian CoefficientPath: "{{crtm_coefficient_path}}" + linear obs operator: + Absorbers: [H2O, O3] # Observation Bias Correction (VarBC) # ----------------------------------- From 4bea8863500cc148bdd1dd40868109ae9063b669 Mon Sep 17 00:00:00 2001 From: Jiarui Dong Date: Fri, 25 Apr 2025 09:36:39 -0400 Subject: [PATCH 109/199] Add initial configures enable for cycle dependent reject lists (#86) This PR adds the initial configures enable for using the different reject lists for the different cycles. The purpose of this PR is to use the JCB conventional chronicle as separate reject lists to remove the bad snow depth reports in the 06/12/18 cycles in China. This PR adds eleven stations from China in the reject list at t00z. This PR also removes the stations in China from reject list on October 1, 2022. This PR contributes https://github.com/NOAA-EMC/GDASApp/pull/1549 --- .../snow/{ => t00z}/adpsfc_snow.yaml | 12 ++- .../snow/{ => t00z}/sfcsno.yaml | 12 ++- .../snow/t06z/adpsfc_snow.yaml | 78 +++++++++++++++++++ .../snow/t06z/sfcsno.yaml | 78 +++++++++++++++++++ parm/jcb-gdas/observation_chronicle/snow/t12z | 1 + parm/jcb-gdas/observation_chronicle/snow/t18z | 1 + 6 files changed, 178 insertions(+), 4 deletions(-) rename parm/jcb-gdas/observation_chronicle/snow/{ => t00z}/adpsfc_snow.yaml (62%) rename parm/jcb-gdas/observation_chronicle/snow/{ => t00z}/sfcsno.yaml (62%) create mode 100644 parm/jcb-gdas/observation_chronicle/snow/t06z/adpsfc_snow.yaml create mode 100644 parm/jcb-gdas/observation_chronicle/snow/t06z/sfcsno.yaml create mode 120000 parm/jcb-gdas/observation_chronicle/snow/t12z create mode 120000 parm/jcb-gdas/observation_chronicle/snow/t18z diff --git a/parm/jcb-gdas/observation_chronicle/snow/adpsfc_snow.yaml b/parm/jcb-gdas/observation_chronicle/snow/t00z/adpsfc_snow.yaml similarity index 62% rename from parm/jcb-gdas/observation_chronicle/snow/adpsfc_snow.yaml rename to parm/jcb-gdas/observation_chronicle/snow/t00z/adpsfc_snow.yaml index c44170e8a..cc3d8d944 100644 --- a/parm/jcb-gdas/observation_chronicle/snow/adpsfc_snow.yaml +++ b/parm/jcb-gdas/observation_chronicle/snow/t00z/adpsfc_snow.yaml @@ -9,12 +9,15 @@ window_option: max # observation type initial configuration # -------------------------------- stations_to_reject: [71120, 71397, 71621, 71727, 71816 # from North America - , 10863, 16179 # from Europe - , 40550, 40580, 40582, 40587, 40592 # Mid East + , 71641, 71976, 71743, 71902 + , 10863, 16179 # from Europe for unrealistic snow presence during summer + , 40550, 40580, 40582, 40587, 40592 # from Mid East for unrealistic snow values , 47005, 47008, 47014, 47016, 47020, 47022, 47025, 47028, 47031, 47035 # from North Korea , 47037, 47039, 47041, 47046, 47050, 47052, 47055, 47058, 47060, 47061 , 47065, 47067, 47068, 47069, 47070, 47069, 47070, 47075, 47090 , 48698, 48830 # from Singapore + # Snow reports with frequent incorrect zero values and snow presence during summer from China + , 50136, 50603, 50953, 50968, 52955, 55696, 56004, 56146, 56357, 56586 # China , 65250] # from Africa # Chronicle of changes for this observation type @@ -26,5 +29,10 @@ chronicles: remove_from_reject_list: [71120, 71397, 71727, 71816] - action_date: "2022-10-01T00:00:00" + justification: 'These stations are okay from this time' + remove_from_reject_list: [50136, 50603, 50953, 50968, 52955, 55696 + , 56004, 56146, 56357, 56586] + +- action_date: "2022-10-01T01:00:00" justification: 'Add to reject these stations' add_to_reject_list: [47095, 47098, 47101, 47102] diff --git a/parm/jcb-gdas/observation_chronicle/snow/sfcsno.yaml b/parm/jcb-gdas/observation_chronicle/snow/t00z/sfcsno.yaml similarity index 62% rename from parm/jcb-gdas/observation_chronicle/snow/sfcsno.yaml rename to parm/jcb-gdas/observation_chronicle/snow/t00z/sfcsno.yaml index c44170e8a..cc3d8d944 100644 --- a/parm/jcb-gdas/observation_chronicle/snow/sfcsno.yaml +++ b/parm/jcb-gdas/observation_chronicle/snow/t00z/sfcsno.yaml @@ -9,12 +9,15 @@ window_option: max # observation type initial configuration # -------------------------------- stations_to_reject: [71120, 71397, 71621, 71727, 71816 # from North America - , 10863, 16179 # from Europe - , 40550, 40580, 40582, 40587, 40592 # Mid East + , 71641, 71976, 71743, 71902 + , 10863, 16179 # from Europe for unrealistic snow presence during summer + , 40550, 40580, 40582, 40587, 40592 # from Mid East for unrealistic snow values , 47005, 47008, 47014, 47016, 47020, 47022, 47025, 47028, 47031, 47035 # from North Korea , 47037, 47039, 47041, 47046, 47050, 47052, 47055, 47058, 47060, 47061 , 47065, 47067, 47068, 47069, 47070, 47069, 47070, 47075, 47090 , 48698, 48830 # from Singapore + # Snow reports with frequent incorrect zero values and snow presence during summer from China + , 50136, 50603, 50953, 50968, 52955, 55696, 56004, 56146, 56357, 56586 # China , 65250] # from Africa # Chronicle of changes for this observation type @@ -26,5 +29,10 @@ chronicles: remove_from_reject_list: [71120, 71397, 71727, 71816] - action_date: "2022-10-01T00:00:00" + justification: 'These stations are okay from this time' + remove_from_reject_list: [50136, 50603, 50953, 50968, 52955, 55696 + , 56004, 56146, 56357, 56586] + +- action_date: "2022-10-01T01:00:00" justification: 'Add to reject these stations' add_to_reject_list: [47095, 47098, 47101, 47102] diff --git a/parm/jcb-gdas/observation_chronicle/snow/t06z/adpsfc_snow.yaml b/parm/jcb-gdas/observation_chronicle/snow/t06z/adpsfc_snow.yaml new file mode 100644 index 000000000..4f4ff7be9 --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/snow/t06z/adpsfc_snow.yaml @@ -0,0 +1,78 @@ +# Instrument metadata +# ------------------- +commissioned: 1970-01-01T00:00:00 + +observer_type: conventional # Type of chronicle to use + +window_option: max + +# observation type initial configuration +# -------------------------------- +stations_to_reject: [71120, 71397, 71621, 71727, 71816 # from North America + , 71641, 71976, 71743, 71902 + , 10863, 16179 # from Europe + , 40550, 40580, 40582, 40587, 40592 # Mid East + , 47005, 47008, 47014, 47016, 47020, 47022, 47025, 47028, 47031, 47035 # from North Korea + , 47037, 47039, 47041, 47046, 47050, 47052, 47055, 47058, 47060, 47061 + , 47065, 47067, 47068, 47069, 47070, 47069, 47070, 47075, 47090 + , 48698, 48830 # from Singapore + # Snow reports with frequent incorrect zero values in 06/12/18z cycles from China + , 50136, 50353, 50468, 50557, 50564, 50603, 50658, 50745, 50756, 50774 # from China + , 50788, 50844, 50854, 50888, 50949, 50953, 50963, 50968, 50978, 50983 + , 51467, 51573, 51644, 51656, 51716, 51747, 51765, 51777, 51828, 51886 + , 52203, 52267, 52323, 52418, 52436, 52533, 52652, 52681, 52713, 52754 + , 52818, 52836, 52866, 52908, 52955, 52983, 53068, 53083, 53149, 53192 + , 53231, 53276, 53336, 53352, 53391, 53463, 53480, 53487, 53502, 53513 + , 53529, 53543, 53564, 53588, 53593, 53614, 53646, 53673, 53698, 53705 + , 53723, 53764, 53772, 53787, 53798, 53845, 53863, 53898, 53915, 53923 + , 53959, 53975, 54026, 54027, 54049, 54094, 54096, 54115, 54135, 54157 + , 54161, 54186, 54208, 54218, 54226, 54236, 54259, 54273, 54292, 54308 + , 54311, 54324, 54337, 54342, 54346, 54374, 54377, 54386, 54401, 54405 + , 54423, 54436, 54471, 54493, 54497, 54511, 54527, 54534, 54539, 54602 + , 54618, 54662, 54715, 54725, 54751, 54753, 54776, 54808, 54823, 54826 + , 54836, 54843, 54857, 54863, 54909, 54916, 54929, 55279, 55472, 55578 + , 55591, 55664, 55696, 55773, 56004, 56033, 56065, 56096, 56106, 56116 + , 56137, 56146, 56172, 56182, 56257, 56357, 56374, 56586, 56691, 57014 + , 57046, 57067, 57071, 57083, 57259, 57265, 57279, 57297, 57378, 57399 + , 57461, 57476, 57494, 57554, 57584, 57655, 57707, 57712, 57731, 57745 + , 57766, 57776, 57793, 57799, 57832, 57845, 57853, 57866, 57902, 57922 + , 58027, 58141, 58150, 58208, 58265, 58345, 58437, 58477, 58506, 58543 + , 58556, 58569, 58633, 58652, 58725, 58730, 58731, 58752 + , 65250] # from Africa + +# Chronicle of changes for this observation type +# ---------------------------------------- +chronicles: + +- action_date: "2021-10-01T00:00:00" + justification: 'These stations are okay from this time' + remove_from_reject_list: [71120, 71397, 71727, 71816] + +- action_date: "2022-10-01T00:00:00" + justification: 'These stations are okay from this time' + remove_from_reject_list: [50136 + , 50353, 50468, 50557, 50564, 50603, 50658, 50745, 50756, 50774 # from China + , 50788, 50844, 50854, 50888, 50949, 50953, 50963, 50968, 50978, 50983 + , 51467, 51573, 51644, 51656, 51716, 51747, 51765, 51777, 51828, 51886 + , 52203, 52267, 52323, 52418, 52436, 52533, 52652, 52681, 52713, 52754 + , 52818, 52836, 52866, 52908, 52955, 52983, 53068, 53083, 53149, 53192 + , 53231, 53276, 53336, 53352, 53391, 53463, 53480, 53487, 53502, 53513 + , 53529, 53543, 53564, 53588, 53593, 53614, 53646, 53673, 53698, 53705 + , 53723, 53764, 53772, 53787, 53798, 53845, 53863, 53898, 53915, 53923 + , 53959, 53975, 54026, 54027, 54049, 54094, 54096, 54115, 54135, 54157 + , 54161, 54186, 54208, 54218, 54226, 54236, 54259, 54273, 54292, 54308 + , 54311, 54324, 54337, 54342, 54346, 54374, 54377, 54386, 54401, 54405 + , 54423, 54436, 54471, 54493, 54497, 54511, 54527, 54534, 54539, 54602 + , 54618, 54662, 54715, 54725, 54751, 54753, 54776, 54808, 54823, 54826 + , 54836, 54843, 54857, 54863, 54909, 54916, 54929, 55279, 55472, 55578 + , 55591, 55664, 55696, 55773, 56004, 56033, 56065, 56096, 56106, 56116 + , 56137, 56146, 56172, 56182, 56257, 56357, 56374, 56586, 56691, 57014 + , 57046, 57067, 57071, 57083, 57259, 57265, 57279, 57297, 57378, 57399 + , 57461, 57476, 57494, 57554, 57584, 57655, 57707, 57712, 57731, 57745 + , 57766, 57776, 57793, 57799, 57832, 57845, 57853, 57866, 57902, 57922 + , 58027, 58141, 58150, 58208, 58265, 58345, 58437, 58477, 58506, 58543 + , 58556, 58569, 58633, 58652, 58725, 58730, 58731, 58752] + +- action_date: "2022-10-01T01:00:00" + justification: 'Add to reject these stations' + add_to_reject_list: [47095, 47098, 47101, 47102] diff --git a/parm/jcb-gdas/observation_chronicle/snow/t06z/sfcsno.yaml b/parm/jcb-gdas/observation_chronicle/snow/t06z/sfcsno.yaml new file mode 100644 index 000000000..4f4ff7be9 --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/snow/t06z/sfcsno.yaml @@ -0,0 +1,78 @@ +# Instrument metadata +# ------------------- +commissioned: 1970-01-01T00:00:00 + +observer_type: conventional # Type of chronicle to use + +window_option: max + +# observation type initial configuration +# -------------------------------- +stations_to_reject: [71120, 71397, 71621, 71727, 71816 # from North America + , 71641, 71976, 71743, 71902 + , 10863, 16179 # from Europe + , 40550, 40580, 40582, 40587, 40592 # Mid East + , 47005, 47008, 47014, 47016, 47020, 47022, 47025, 47028, 47031, 47035 # from North Korea + , 47037, 47039, 47041, 47046, 47050, 47052, 47055, 47058, 47060, 47061 + , 47065, 47067, 47068, 47069, 47070, 47069, 47070, 47075, 47090 + , 48698, 48830 # from Singapore + # Snow reports with frequent incorrect zero values in 06/12/18z cycles from China + , 50136, 50353, 50468, 50557, 50564, 50603, 50658, 50745, 50756, 50774 # from China + , 50788, 50844, 50854, 50888, 50949, 50953, 50963, 50968, 50978, 50983 + , 51467, 51573, 51644, 51656, 51716, 51747, 51765, 51777, 51828, 51886 + , 52203, 52267, 52323, 52418, 52436, 52533, 52652, 52681, 52713, 52754 + , 52818, 52836, 52866, 52908, 52955, 52983, 53068, 53083, 53149, 53192 + , 53231, 53276, 53336, 53352, 53391, 53463, 53480, 53487, 53502, 53513 + , 53529, 53543, 53564, 53588, 53593, 53614, 53646, 53673, 53698, 53705 + , 53723, 53764, 53772, 53787, 53798, 53845, 53863, 53898, 53915, 53923 + , 53959, 53975, 54026, 54027, 54049, 54094, 54096, 54115, 54135, 54157 + , 54161, 54186, 54208, 54218, 54226, 54236, 54259, 54273, 54292, 54308 + , 54311, 54324, 54337, 54342, 54346, 54374, 54377, 54386, 54401, 54405 + , 54423, 54436, 54471, 54493, 54497, 54511, 54527, 54534, 54539, 54602 + , 54618, 54662, 54715, 54725, 54751, 54753, 54776, 54808, 54823, 54826 + , 54836, 54843, 54857, 54863, 54909, 54916, 54929, 55279, 55472, 55578 + , 55591, 55664, 55696, 55773, 56004, 56033, 56065, 56096, 56106, 56116 + , 56137, 56146, 56172, 56182, 56257, 56357, 56374, 56586, 56691, 57014 + , 57046, 57067, 57071, 57083, 57259, 57265, 57279, 57297, 57378, 57399 + , 57461, 57476, 57494, 57554, 57584, 57655, 57707, 57712, 57731, 57745 + , 57766, 57776, 57793, 57799, 57832, 57845, 57853, 57866, 57902, 57922 + , 58027, 58141, 58150, 58208, 58265, 58345, 58437, 58477, 58506, 58543 + , 58556, 58569, 58633, 58652, 58725, 58730, 58731, 58752 + , 65250] # from Africa + +# Chronicle of changes for this observation type +# ---------------------------------------- +chronicles: + +- action_date: "2021-10-01T00:00:00" + justification: 'These stations are okay from this time' + remove_from_reject_list: [71120, 71397, 71727, 71816] + +- action_date: "2022-10-01T00:00:00" + justification: 'These stations are okay from this time' + remove_from_reject_list: [50136 + , 50353, 50468, 50557, 50564, 50603, 50658, 50745, 50756, 50774 # from China + , 50788, 50844, 50854, 50888, 50949, 50953, 50963, 50968, 50978, 50983 + , 51467, 51573, 51644, 51656, 51716, 51747, 51765, 51777, 51828, 51886 + , 52203, 52267, 52323, 52418, 52436, 52533, 52652, 52681, 52713, 52754 + , 52818, 52836, 52866, 52908, 52955, 52983, 53068, 53083, 53149, 53192 + , 53231, 53276, 53336, 53352, 53391, 53463, 53480, 53487, 53502, 53513 + , 53529, 53543, 53564, 53588, 53593, 53614, 53646, 53673, 53698, 53705 + , 53723, 53764, 53772, 53787, 53798, 53845, 53863, 53898, 53915, 53923 + , 53959, 53975, 54026, 54027, 54049, 54094, 54096, 54115, 54135, 54157 + , 54161, 54186, 54208, 54218, 54226, 54236, 54259, 54273, 54292, 54308 + , 54311, 54324, 54337, 54342, 54346, 54374, 54377, 54386, 54401, 54405 + , 54423, 54436, 54471, 54493, 54497, 54511, 54527, 54534, 54539, 54602 + , 54618, 54662, 54715, 54725, 54751, 54753, 54776, 54808, 54823, 54826 + , 54836, 54843, 54857, 54863, 54909, 54916, 54929, 55279, 55472, 55578 + , 55591, 55664, 55696, 55773, 56004, 56033, 56065, 56096, 56106, 56116 + , 56137, 56146, 56172, 56182, 56257, 56357, 56374, 56586, 56691, 57014 + , 57046, 57067, 57071, 57083, 57259, 57265, 57279, 57297, 57378, 57399 + , 57461, 57476, 57494, 57554, 57584, 57655, 57707, 57712, 57731, 57745 + , 57766, 57776, 57793, 57799, 57832, 57845, 57853, 57866, 57902, 57922 + , 58027, 58141, 58150, 58208, 58265, 58345, 58437, 58477, 58506, 58543 + , 58556, 58569, 58633, 58652, 58725, 58730, 58731, 58752] + +- action_date: "2022-10-01T01:00:00" + justification: 'Add to reject these stations' + add_to_reject_list: [47095, 47098, 47101, 47102] diff --git a/parm/jcb-gdas/observation_chronicle/snow/t12z b/parm/jcb-gdas/observation_chronicle/snow/t12z new file mode 120000 index 000000000..e490a10cf --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/snow/t12z @@ -0,0 +1 @@ +t06z \ No newline at end of file diff --git a/parm/jcb-gdas/observation_chronicle/snow/t18z b/parm/jcb-gdas/observation_chronicle/snow/t18z new file mode 120000 index 000000000..e490a10cf --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/snow/t18z @@ -0,0 +1 @@ +t06z \ No newline at end of file From f2ddf811e9d9897905d50aa6800968399c7d21cf Mon Sep 17 00:00:00 2001 From: Jiarui Dong Date: Mon, 5 May 2025 08:37:40 -0400 Subject: [PATCH 110/199] Add fraction of ice to filter out obs with ice presence (#115) This PR adds the fraction of ice to filter out the observations if ice is present, because the increments are keeping to be added at locations with persistent positive O-F. This PR also adds error inflation to station observations (sfcsno and snocvr) from @tsga --- .../snow/fv3jedi_snow_ensmean.yaml.j2 | 2 +- .../model/snow/snow_background.yaml.j2 | 2 +- .../observations/snow/ims_snow.yaml.j2 | 5 ++ .../jcb-gdas/observations/snow/sfcsno.yaml.j2 | 52 ++++++++++++++++++- .../observations/snow/snocvr_snow.yaml.j2 | 52 ++++++++++++++++++- 5 files changed, 109 insertions(+), 4 deletions(-) diff --git a/parm/jcb-gdas/algorithm/snow/fv3jedi_snow_ensmean.yaml.j2 b/parm/jcb-gdas/algorithm/snow/fv3jedi_snow_ensmean.yaml.j2 index bf13c15b1..b48929e12 100644 --- a/parm/jcb-gdas/algorithm/snow/fv3jedi_snow_ensmean.yaml.j2 +++ b/parm/jcb-gdas/algorithm/snow/fv3jedi_snow_ensmean.yaml.j2 @@ -16,7 +16,7 @@ ensemble: template: datetime: '{{ snow_background_time_iso }}' filetype: fms restart - state variables: [snodl,vtype,slmsk,orog_filt,stc,sheleg] + state variables: [snodl,vtype,slmsk,orog_filt,stc,sheleg,fice] datapath: ./bkg/mem%mem%/ filename_sfcd: '{{ snow_background_time_fv3 }}.sfc_data.nc' filename_cplr: '{{ snow_background_time_fv3 }}.coupler.res' diff --git a/parm/jcb-gdas/model/snow/snow_background.yaml.j2 b/parm/jcb-gdas/model/snow/snow_background.yaml.j2 index 737d5e3f5..b993a3eb8 100644 --- a/parm/jcb-gdas/model/snow/snow_background.yaml.j2 +++ b/parm/jcb-gdas/model/snow/snow_background.yaml.j2 @@ -2,7 +2,7 @@ datapath: {{ snow_background_path }} filetype: fms restart skip coupler file: true datetime: '{{ snow_background_time_iso }}' -state variables: [snodl,vtype,slmsk,sheleg,orog_filt] +state variables: [snodl,vtype,slmsk,sheleg,orog_filt,fice] filename_sfcd: '{{ snow_background_time_fv3 }}.sfc_data.nc' filename_cplr: '{{ snow_background_time_fv3 }}.coupler.res' filename_orog: '{{ snow_orog_prefix }}_oro_data.nc' diff --git a/parm/jcb-gdas/observations/snow/ims_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/ims_snow.yaml.j2 index 612f2d73a..fd7ca7ba2 100644 --- a/parm/jcb-gdas/observations/snow/ims_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/ims_snow.yaml.j2 @@ -41,6 +41,11 @@ name: GeoVaLs/slmsk minvalue: 0.5 maxvalue: 1.5 + - filter: Domain Check # land only, no sea ice + where: + - variable: + name: GeoVaLs/fraction_of_ice + maxvalue: 0.0 - filter: RejectList # no land-ice where: - variable: diff --git a/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 b/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 index ce3711075..827a8899f 100644 --- a/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 @@ -134,6 +134,11 @@ flag: invalid_elevation ignore: rejected observations - name: reject + - filter: Domain Check # land only, no sea ice + where: + - variable: + name: GeoVaLs/fraction_of_ice + maxvalue: 0.0 - filter: RejectList # no land-ice where: - variable: @@ -148,12 +153,55 @@ - filter: Difference Check # elevation check reference: MetaData/stationElevation value: GeoVaLs/filtered_orography - threshold: 200. + threshold: 800. actions: - name: set flag: elevation_bkgdiff ignore: rejected observations - name: reject + # Add inflate error characterized by a function of elevation difference + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/elevation_diff + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: [MetaData/stationElevation, GeoVaLs/filtered_orography] + coefficients: [1., -1.] + total exponent: 2 + total coefficient: 0.0000015625 #1/(800*800) + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/elevation_diff_exp + type: float + function: + name: ObsFunction/Exponential + options: + variables: [DerivedMetaData/elevation_diff] + coeffA: 1. + coeffB: -1. + coeffC: 0. + coeffD: 1. + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/elevation_diff_exp_inv + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: [DerivedMetaData/elevation_diff_exp] + total exponent: -1. + - filter: Perform Action + filter variables: + - name: totalSnowDepth + where: + - variable: + name: ObsValue/totalSnowDepth + minvalue: 0. + action: + name: inflate error + inflation variable: DerivedMetaData/elevation_diff_exp_inv # 2.0 - filter: BlackList where: - variable: @@ -169,6 +217,8 @@ filter variables: - name: totalSnowDepth threshold: 6.25 + absolute threshold: 250 + bias correction parameter: 1.0 actions: - name: set flag: background_check diff --git a/parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 index ebef03edf..144ecbbde 100644 --- a/parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 @@ -131,6 +131,11 @@ flag: invalid_elevation ignore: rejected observations - name: reject + - filter: Domain Check # land only, no sea ice + where: + - variable: + name: GeoVaLs/fraction_of_ice + maxvalue: 0.0 - filter: RejectList # no land-ice where: - variable: @@ -145,17 +150,62 @@ - filter: Difference Check # elevation check reference: MetaData/stationElevation value: GeoVaLs/filtered_orography - threshold: 200. + threshold: 800. actions: - name: set flag: elevation_bkgdiff ignore: rejected observations - name: reject + # Add inflate error characterized by a function of elevation difference + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/elevation_diff + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: [MetaData/stationElevation, GeoVaLs/filtered_orography] + coefficients: [1., -1.] + total exponent: 2 + total coefficient: 0.0000015625 #1/(800*800) + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/elevation_diff_exp + type: float + function: + name: ObsFunction/Exponential + options: + variables: [DerivedMetaData/elevation_diff] + coeffA: 1. + coeffB: -1. + coeffC: 0. + coeffD: 1. + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/elevation_diff_exp_inv + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: [DerivedMetaData/elevation_diff_exp] + total exponent: -1. + - filter: Perform Action + filter variables: + - name: totalSnowDepth + where: + - variable: + name: ObsValue/totalSnowDepth + minvalue: 0. + action: + name: inflate error + inflation variable: DerivedMetaData/elevation_diff_exp_inv # 2.0 obs post filters: - filter: Background Check # gross error check filter variables: - name: totalSnowDepth threshold: 6.25 + absolute threshold: 250 + bias correction parameter: 1.0 actions: - name: set flag: background_check From b36e9430dfee6d79575340be11c069b9d5c817b6 Mon Sep 17 00:00:00 2001 From: JohnSteffen-NOAA <77806417+JohnSteffen-NOAA@users.noreply.github.com> Date: Mon, 5 May 2025 08:44:01 -0400 Subject: [PATCH 111/199] Bugfixes for marine yamls (#116) This PR implements several bugfixes and typo corrections in the marine yamls: 1. The error coefficients are added back to both Argo yamls 2. The background check post filter in the sst_generic yamls is removed 3. A sst_generic_geo.yaml.j2 file is created to place the geostationary SST retrievals in monitoring mode Co-authored-by: Guillaume Vernieres --- .../marine/insitu_profile_bathy.yaml.j2 | 2 +- .../marine/insitu_salt_profile_argo.yaml.j2 | 16 ++++ .../marine/insitu_temp_profile_argo.yaml.j2 | 17 ++++ .../marine/sst_abi_g16_l3c.yaml.j2 | 2 +- .../marine/sst_abi_g17_l3c.yaml.j2 | 2 +- .../marine/sst_ahi_h08_l3c.yaml.j2 | 2 +- .../observations/marine/sst_generic.yaml.j2 | 2 - .../marine/sst_generic_geo.yaml.j2 | 80 +++++++++++++++++++ 8 files changed, 117 insertions(+), 6 deletions(-) create mode 100644 parm/jcb-gdas/observations/marine/sst_generic_geo.yaml.j2 diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 index 2f811f062..b622fd06c 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 @@ -9,7 +9,7 @@ type: H5File obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" simulated variables: [waterTemperature] - observed variable: [waterTemperature] + observed variables: [waterTemperature] io pool: max pool size: 1 obs operator: diff --git a/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 index 0448247ce..12d75fa2f 100644 --- a/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 @@ -31,6 +31,7 @@ # The QC filters used here are based on the document by IODE that can be found at # https://cdn.ioos.noaa.gov/media/2017/12/recommendations_in_situ_data_real_time_qc.pdf #------------------------------------------------------------------------------- + obs filters: # land check @@ -45,6 +46,7 @@ #----------------------------------------------------------------------------- ### Global range test #----------------------------------------------------------------------------- + - filter: Bounds Check filter variables: [{name: salinity}] minvalue: 2.0 @@ -123,6 +125,7 @@ #### Northwestern shelves #----------------------------------------------------------------------------- + - filter: Bounds Check filter variables: [{name: salinity}] minvalue: 0.0 @@ -139,6 +142,7 @@ #### Southwestern shelves #----------------------------------------------------------------------------- + - filter: Bounds Check filter variables: [{name: salinity}] minvalue: 0.0 @@ -155,6 +159,7 @@ #### Arctic Ocean #----------------------------------------------------------------------------- + - filter: Bounds Check filter variables: [{name: salinity}] minvalue: 2.0 @@ -169,6 +174,17 @@ threshold: 5.0 absolute threshold: 5.0 + - filter: Perform Action + action: + name: assign error + error function: + name: ObsFunction/LinearCombination + options: + variables: + - ObsError/salinity + coefs: + - 1000.0 + obs localizations: - localization method: Rossby base value: 100.0e3 diff --git a/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 index ff640d832..898f10df1 100644 --- a/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 @@ -27,6 +27,7 @@ # The QC filters used here are based on the document by IODE that can be found at # https://cdn.ioos.noaa.gov/media/2017/12/recommendations_in_situ_data_real_time_qc.pdf #------------------------------------------------------------------------------- + obs filters: # land check @@ -41,6 +42,7 @@ #------------------------------------------------------------------------------- ### Global range test #----------------------------------------------------------------------------- + - filter: Bounds Check filter variables: [{name: waterTemperature}] minvalue: -2.5 @@ -119,6 +121,7 @@ #### Northwestern shelves #----------------------------------------------------------------------------- + - filter: Bounds Check filter variables: [{name: waterTemperature}] minvalue: -2.0 @@ -132,8 +135,10 @@ name: MetaData/longitude minvalue: -20 maxvalue: 10 + #### Southwestern shelves #----------------------------------------------------------------------------- + - filter: Bounds Check filter variables: [{name: waterTemperature}] minvalue: -2.0 @@ -150,6 +155,7 @@ #### Arctic Ocean #----------------------------------------------------------------------------- + - filter: Bounds Check filter variables: [{name: waterTemperature}] minvalue: -1.92 @@ -164,6 +170,17 @@ threshold: 5.0 absolute threshold: 5.0 + - filter: Perform Action + action: + name: assign error + error function: + name: ObsFunction/LinearCombination + options: + variables: + - ObsError/waterTemperature + coefs: + - 1000.0 + obs localizations: - localization method: Rossby base value: 100.0e3 diff --git a/parm/jcb-gdas/observations/marine/sst_abi_g16_l3c.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_abi_g16_l3c.yaml.j2 index 5298b9a81..b3649a670 120000 --- a/parm/jcb-gdas/observations/marine/sst_abi_g16_l3c.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sst_abi_g16_l3c.yaml.j2 @@ -1 +1 @@ -sst_generic.yaml.j2 \ No newline at end of file +sst_generic_geo.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/sst_abi_g17_l3c.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_abi_g17_l3c.yaml.j2 index 5298b9a81..b3649a670 120000 --- a/parm/jcb-gdas/observations/marine/sst_abi_g17_l3c.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sst_abi_g17_l3c.yaml.j2 @@ -1 +1 @@ -sst_generic.yaml.j2 \ No newline at end of file +sst_generic_geo.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/sst_ahi_h08_l3c.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_ahi_h08_l3c.yaml.j2 index 5298b9a81..b3649a670 120000 --- a/parm/jcb-gdas/observations/marine/sst_ahi_h08_l3c.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sst_ahi_h08_l3c.yaml.j2 @@ -1 +1 @@ -sst_generic.yaml.j2 \ No newline at end of file +sst_generic_geo.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 index b0755aac8..8c02ef1ad 100644 --- a/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 @@ -56,8 +56,6 @@ {% endif %} obs post filters: - - filter: Background Check - absolute threshold: 5.0 - filter: Perform Action # Pamlico Sound & Chesapeake Bay action: name: accept diff --git a/parm/jcb-gdas/observations/marine/sst_generic_geo.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_generic_geo.yaml.j2 new file mode 100644 index 000000000..19a76ab93 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/sst_generic_geo.yaml.j2 @@ -0,0 +1,80 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaSurfaceTemperature] + io pool: + max pool size: 1 + get values: + time interpolation: linear + obs operator: + name: Identity + observation alias file: obsop_name_map.yaml + obs error: + covariance model: diagonal + + obs pre filters: + - filter: Bounds Check + minvalue: -1.2 + maxvalue: 41.0 + - filter: Domain Check + where: + - variable: {name: ObsError/seaSurfaceTemperature} + minvalue: 1.0e-8 + + obs prior filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + minvalue: 0.9 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_ice_area_fraction} + maxvalue: 1.0e-5 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_surface_temperature} + minvalue: -1.2 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/distance_from_coast} + minvalue: 50e3 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} + - filter: Domain Check + action: + name: passivate + where: + - variable: {name: GeoVaLs/sea_surface_temperature} + maxvalue: -4.0 + + + obs post filters: + - filter: Perform Action # Pamlico Sound & Chesapeake Bay + action: + name: accept + where: + - variable: + name: MetaData/latitude + minvalue: 34.5 + maxvalue: 39.7 + - variable: + name: MetaData/longitude + minvalue: -76.88 + maxvalue: -74.5 +# TODO: var fails when bkg chk filter is last +# - filter: Background Check +# absolute threshold: 5.0 From f2f68fb4c3c9a06b9f6480be88d30aa6ad3352c6 Mon Sep 17 00:00:00 2001 From: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com> Date: Mon, 5 May 2025 16:33:30 -0400 Subject: [PATCH 112/199] localization bug fixes for marine observations (#117) --- .../marine/insitu_salt_profile_argo.yaml.j2 | 8 -------- .../marine/insitu_temp_profile_argo.yaml.j2 | 8 -------- .../observations/marine/sst_generic_geo.yaml.j2 | 13 ++++++------- 3 files changed, 6 insertions(+), 23 deletions(-) diff --git a/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 index 12d75fa2f..b5772500e 100644 --- a/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 @@ -184,14 +184,6 @@ - ObsError/salinity coefs: - 1000.0 - - obs localizations: - - localization method: Rossby - base value: 100.0e3 - rossby mult: 1.0 - min grid mult: 2.0 - min value: 200.0e3 - max value: 900.0e3 {% if letkf_app | default(false) %} obs localizations: - localization method: Rossby diff --git a/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 index 898f10df1..ebd55d2ba 100644 --- a/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 @@ -180,14 +180,6 @@ - ObsError/waterTemperature coefs: - 1000.0 - - obs localizations: - - localization method: Rossby - base value: 100.0e3 - rossby mult: 1.0 - min grid mult: 2.0 - min value: 200.0e3 - max value: 900.0e3 {% if letkf_app | default(false) %} obs localizations: - localization method: Rossby diff --git a/parm/jcb-gdas/observations/marine/sst_generic_geo.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_generic_geo.yaml.j2 index 19a76ab93..35a7d2813 100644 --- a/parm/jcb-gdas/observations/marine/sst_generic_geo.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sst_generic_geo.yaml.j2 @@ -45,6 +45,12 @@ where: - variable: {name: GeoVaLs/distance_from_coast} minvalue: 50e3 + - filter: Domain Check + action: + name: passivate + where: + - variable: {name: GeoVaLs/sea_surface_temperature} + maxvalue: -4.0 {% if letkf_app | default(false) %} obs localizations: - localization method: Rossby @@ -54,13 +60,6 @@ min value: 200.0e3 max value: 900.0e3 {% endif %} - - filter: Domain Check - action: - name: passivate - where: - - variable: {name: GeoVaLs/sea_surface_temperature} - maxvalue: -4.0 - obs post filters: - filter: Perform Action # Pamlico Sound & Chesapeake Bay From eb3ed8c6e8f31932e6e77fdd7a54b43dffb9c727 Mon Sep 17 00:00:00 2001 From: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com> Date: Wed, 7 May 2025 07:11:28 -0400 Subject: [PATCH 113/199] update interpolation section of gsibec templates (#113) --- ...ackground_error_hybrid_gsibec_bump.yaml.j2 | 22 +++++++++++++------ ...ere_background_error_static_gsibec.yaml.j2 | 22 +++++++++++++------ 2 files changed, 30 insertions(+), 14 deletions(-) diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 index 80ac8ed98..88ce07287 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 @@ -12,13 +12,21 @@ components: processor layout y direction: {{atmosphere_layout_gsib_y}} debugging mode: false saber outer blocks: - - saber block name: gsi interpolation to model grid - gsi akbk: {{atmosphere_fv3jedi_files_path}}/akbk.nc4 - gsi error covariance file: {{atmosphere_gsibec_path}}/gsi-coeffs-gfs-global.nc4 - gsi berror namelist file: {{atmosphere_gsibec_path}}/gfs_gsi_global.nml - processor layout x direction: {{atmosphere_layout_gsib_x}} - processor layout y direction: {{atmosphere_layout_gsib_y}} - debugging mode: false + - saber block name: interpolation + inner geometry: + function space: StructuredColumns + custom grid matching gsi: + type: gaussian + lats: {{ atmosphere_npy_ges + 1 }} + {% set atmosphere_lons = (atmosphere_npx_ges - 1) * 2 %} + lons: {{ atmosphere_lons }} + custom partitioner matching gsi: + bands: {{ atmosphere_layout_gsib_y }} + halo: 1 + forward interpolator: + local interpolator type: oops unstructured grid interpolator + inverse interpolator: + local interpolator type: oops unstructured grid interpolator state variables to inverse: [eastward_wind,northward_wind,air_temperature,air_pressure_at_surface, water_vapor_mixing_ratio_wrt_moist_air,cloud_liquid_ice,cloud_liquid_water, ozone_mass_mixing_ratio] diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_gsibec.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_gsibec.yaml.j2 index d77b49e49..73983d706 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_gsibec.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_gsibec.yaml.j2 @@ -10,13 +10,21 @@ saber central block: processor layout y direction: {{atmosphere_layout_gsib_y}} debugging mode: false saber outer blocks: -- saber block name: gsi interpolation to model grid - gsi akbk: {{atmosphere_fv3jedi_files_path}}/akbk.nc4 - gsi error covariance file: {{atmosphere_gsibec_path}}/gsi-coeffs-gfs-global.nc4 - gsi berror namelist file: {{atmosphere_gsibec_path}}/gfs_gsi_global.nml - processor layout x direction: {{atmosphere_layout_gsib_x}} - processor layout y direction: {{atmosphere_layout_gsib_y}} - debugging mode: false +- saber block name: interpolation + inner geometry: + function space: StructuredColumns + custom grid matching gsi: + type: gaussian + lats: {{ atmosphere_npy_ges + 1 }} + {% set atmosphere_lons = (atmosphere_npx_ges - 1) * 2 %} + lons: {{ atmosphere_lons }} + custom partitioner matching gsi: + bands: {{ atmosphere_layout_gsib_y }} + halo: 1 + forward interpolator: + local interpolator type: oops unstructured grid interpolator + inverse interpolator: + local interpolator type: oops unstructured grid interpolator state variables to inverse: [eastward_wind,northward_wind,air_temperature,air_pressure_at_surface, water_vapor_mixing_ratio_wrt_moist_air,cloud_liquid_ice,cloud_liquid_water, ozone_mass_mixing_ratio] From 3a74b42d10f6e22c9c22cd720fd81458f9909854 Mon Sep 17 00:00:00 2001 From: Jiarui Dong Date: Wed, 7 May 2025 13:34:05 -0400 Subject: [PATCH 114/199] Add snocvr obs entry for realtime parallel experiments and operations (#119) This PR renames the current `snocvr` to `madis` showing the data source for the retrospective experiments, and adds GTS based `snocvr` obs entry for JEDI based snow DA in real-time parallels and operations. This PR also adds the `observation_chronicle` to control the start/end time of the observations from `madis_snow` and `snocvr` for the US snow data. This PR contributes to https://github.com/NOAA-EMC/GDASApp/pull/1675 This PR resolves https://github.com/NOAA-EMC/GDASApp/issues/1663 --- .../algorithm/snow/snow_obs_staging.yaml.j2 | 6 +- .../snow/t00z/madis_snow.yaml | 15 ++ .../snow/t00z/snocvr.yaml | 14 + .../snow/t06z/madis_snow.yaml | 15 ++ .../snow/t06z/snocvr.yaml | 14 + .../observations/snow/adpsfc_snow.yaml.j2 | 123 +++++++-- .../observations/snow/madis_snow.yaml.j2 | 192 ++++++++++++++ .../jcb-gdas/observations/snow/sfcsno.yaml.j2 | 4 +- .../jcb-gdas/observations/snow/snocvr.yaml.j2 | 243 ++++++++++++++++++ 9 files changed, 604 insertions(+), 22 deletions(-) create mode 100644 parm/jcb-gdas/observation_chronicle/snow/t00z/madis_snow.yaml create mode 100644 parm/jcb-gdas/observation_chronicle/snow/t00z/snocvr.yaml create mode 100644 parm/jcb-gdas/observation_chronicle/snow/t06z/madis_snow.yaml create mode 100644 parm/jcb-gdas/observation_chronicle/snow/t06z/snocvr.yaml create mode 100644 parm/jcb-gdas/observations/snow/madis_snow.yaml.j2 create mode 100644 parm/jcb-gdas/observations/snow/snocvr.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/snow/snow_obs_staging.yaml.j2 b/parm/jcb-gdas/algorithm/snow/snow_obs_staging.yaml.j2 index 778cfd1e5..f50b5a911 100644 --- a/parm/jcb-gdas/algorithm/snow/snow_obs_staging.yaml.j2 +++ b/parm/jcb-gdas/algorithm/snow/snow_obs_staging.yaml.j2 @@ -5,9 +5,9 @@ copy_opt: {% if use_observer(observation_from_jcb) %} {% if not observation_from_jcb == 'ims_snow' %} # skip if ims_snow since it is special - {% if observation_from_jcb == 'snocvr_snow' %} - # snocvr_snow is a .nc file, TODO rename this eventually? - - ['{{snow_obsdataroot_path}}/{{snow_obsdatain_prefix}}{{observation_from_jcb}}.nc4', '{{snow_obsdatain_path}}'] + {% if observation_from_jcb == 'madis_snow' %} + # madis_snow is a .nc file with filename 'snocvr_snow' in global GDA. + - ['{{snow_obsdataroot_path}}/{{snow_obsdatain_prefix}}snocvr_snow.nc4', '{{snow_obsdatain_path}}'] {% else %} - ['{{snow_obsdataroot_path}}/{{snow_obsdatain_prefix}}{{observation_from_jcb}}{{snow_obsdatain_suffix}}', '{{snow_obsdatain_path}}'] {% endif %} diff --git a/parm/jcb-gdas/observation_chronicle/snow/t00z/madis_snow.yaml b/parm/jcb-gdas/observation_chronicle/snow/t00z/madis_snow.yaml new file mode 100644 index 000000000..1e9fd35e6 --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/snow/t00z/madis_snow.yaml @@ -0,0 +1,15 @@ +# Instrument metadata +# ------------------- +commissioned: 2021-01-01T00:00:00 +decommissioned: 2023-05-31T18:00:00 + +observer_type: conventional # Type of chronicle to use + +window_option: max + +# observation type initial configuration +# -------------------------------- +stations_to_reject: [fakelist] + +# Chronicle of changes for this observation type +# ---------------------------------------- diff --git a/parm/jcb-gdas/observation_chronicle/snow/t00z/snocvr.yaml b/parm/jcb-gdas/observation_chronicle/snow/t00z/snocvr.yaml new file mode 100644 index 000000000..102dc2c3a --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/snow/t00z/snocvr.yaml @@ -0,0 +1,14 @@ +# Instrument metadata +# ------------------- +commissioned: 2023-06-01T00:00:00 + +observer_type: conventional # Type of chronicle to use + +window_option: max + +# observation type initial configuration +# -------------------------------- +stations_to_reject: [fakelist] + +# Chronicle of changes for this observation type +# ---------------------------------------- diff --git a/parm/jcb-gdas/observation_chronicle/snow/t06z/madis_snow.yaml b/parm/jcb-gdas/observation_chronicle/snow/t06z/madis_snow.yaml new file mode 100644 index 000000000..0fed1104d --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/snow/t06z/madis_snow.yaml @@ -0,0 +1,15 @@ +# Instrument metadata +# ------------------- +commissioned: 2021-01-01T06:00:00 +decommissioned: 2023-05-31T18:00:00 + +observer_type: conventional # Type of chronicle to use + +window_option: max + +# observation type initial configuration +# -------------------------------- +stations_to_reject: [fakelist] + +# Chronicle of changes for this observation type +# ---------------------------------------- diff --git a/parm/jcb-gdas/observation_chronicle/snow/t06z/snocvr.yaml b/parm/jcb-gdas/observation_chronicle/snow/t06z/snocvr.yaml new file mode 100644 index 000000000..06f123c16 --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/snow/t06z/snocvr.yaml @@ -0,0 +1,14 @@ +# Instrument metadata +# ------------------- +commissioned: 2023-06-01T06:00:00 + +observer_type: conventional # Type of chronicle to use + +window_option: max + +# observation type initial configuration +# -------------------------------- +stations_to_reject: [fakelist] + +# Chronicle of changes for this observation type +# ---------------------------------------- diff --git a/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 index 0850057bb..4cb4b3306 100644 --- a/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 @@ -6,8 +6,9 @@ name: adpsfc_snow obsdatain: engine: - type: H5File + type: bufr obsfile: "{{snow_obsdatain_path}}/{{snow_obsdatain_prefix}}{{observation_from_jcb}}{{snow_obsdatain_suffix}}" + mapping file: "{{snow_obsdatain_path}}/bufr_adpsfc_mapping.yaml" missing file action: warn obsgrouping: group variables: [stationIdentification] @@ -25,12 +26,37 @@ components: - name: Identity - name: BackgroundErrorIdentity - obs error: - covariance model: diagonal + linear obs operator: + name: Identity + # # Observation Filters (QC) # ------------------------ obs pre filters: + - filter: Create Diagnostic Flags + flags: + - name: missing_snowdepth + initial value: false + - name: missing_elevation + initial value: false + - name: temporal_thinning + initial value: false + - name: invalid_snowdepth + initial value: false + - name: invalid_elevation + initial value: false + - name: land_check + initial value: false + - name: landice_check + initial value: false + - name: elevation_bkgdiff + initial value: false + - name: rejectlist + initial value: false + - name: background_check + initial value: false + - name: buddy_check + initial value: false - filter: Perform Action filter variables: - name: totalSnowDepth @@ -45,52 +71,109 @@ - name: BkgError/totalSnowDepth_background_error type: float value: 30.0 + - filter: Domain Check + where: + - variable: + name: ObsValue/totalSnowDepth + value: is_valid + actions: + - name: set + flag: missing_snowdepth + ignore: rejected observations + - name: reject + - filter: Domain Check + where: + - variable: + name: MetaData/stationElevation + value: is_valid + actions: + - name: set + flag: missing_elevation + ignore: rejected observations + - name: reject + - filter: Temporal Thinning + min_spacing: '{{ window_length }}' + seed_time: '{{ snow_background_time_iso }}' + category_variable: + name: MetaData/stationIdentification + actions: + - name: set + flag: temporal_thinning + ignore: rejected observations + - name: reject obs prior filters: - filter: Bounds Check filter variables: - name: totalSnowDepth minvalue: 0.0 maxvalue: 20000.0 - action: - name: reject - - filter: Domain Check - where: - - variable: - name: MetaData/stationElevation - value: is_valid + actions: + - name: set + flag: invalid_snowdepth + ignore: rejected observations + - name: reject - filter: Domain Check # land only where: - variable: name: GeoVaLs/slmsk minvalue: 0.5 maxvalue: 1.5 + actions: + - name: set + flag: land_check + ignore: rejected observations + - name: reject + - filter: Domain Check + where: + - variable: + name: MetaData/stationElevation + minvalue: -200.0 + maxvalue: 9900.0 + actions: + - name: set + flag: invalid_elevation + ignore: rejected observations + - name: reject - filter: RejectList # no land-ice where: - variable: name: GeoVaLs/vtype minvalue: 14.5 maxvalue: 15.5 + actions: + - name: set + flag: landice_check + ignore: rejected observations + - name: reject - filter: Difference Check # elevation check reference: MetaData/stationElevation value: GeoVaLs/filtered_orography threshold: 200. + actions: + - name: set + flag: elevation_bkgdiff + ignore: rejected observations + - name: reject - filter: BlackList where: - variable: name: MetaData/stationIdentification is_in: {{ get_conventional_rejected_stations(observation_from_jcb) }} + actions: + - name: set + flag: rejectlist + ignore: rejected observations + - name: reject obs post filters: - filter: Background Check # gross error check filter variables: - name: totalSnowDepth threshold: 6.25 - action: - name: reject - - filter: Temporal Thinning - min_spacing: '{{ window_length }}' - seed_time: '{{ snow_background_time_iso }}' - category_variable: - name: MetaData/stationIdentification + actions: + - name: set + flag: background_check + ignore: rejected observations + - name: reject - filter: Met Office Buddy Check filter variables: - name: totalSnowDepth @@ -114,3 +197,9 @@ damping_factor_1: 1.0 damping_factor_2: 1.0 background_error_group: BkgError + actions: + - name: set + flag: buddy_check + ignore: rejected observations + - name: reject + diff --git a/parm/jcb-gdas/observations/snow/madis_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/madis_snow.yaml.j2 new file mode 100644 index 000000000..f13df6303 --- /dev/null +++ b/parm/jcb-gdas/observations/snow/madis_snow.yaml.j2 @@ -0,0 +1,192 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: madis_snow + obsdatain: + engine: + type: H5File + obsfile: "{{snow_obsdatain_path}}/{{snow_obsdatain_prefix}}snocvr_snow.nc4" + missing file action: warn + obsgrouping: + group variables: [stationIdentification] + obsdataout: + engine: + type: H5File + obsfile: "{{snow_obsdataout_path}}/{{snow_obsdataout_prefix}}{{observation_from_jcb}}{{snow_obsdataout_suffix}}" + simulated variables: [totalSnowDepth] + # + + # Observation Operator + # -------------------- + obs operator: + name: Composite + components: + - name: Identity + - name: BackgroundErrorIdentity + linear obs operator: + name: Identity + # + + # Observation Filters (QC) + # ------------------------ + obs pre filters: + - filter: Create Diagnostic Flags + flags: + - name: missing_snowdepth + initial value: false + - name: missing_elevation + initial value: false + - name: temporal_thinning + initial value: false + - name: invalid_snowdepth + initial value: false + - name: invalid_elevation + initial value: false + - name: land_check + initial value: false + - name: landice_check + initial value: false + - name: elevation_bkgdiff + initial value: false + - name: background_check + initial value: false + - name: buddy_check + initial value: false + - filter: Perform Action + filter variables: + - name: totalSnowDepth + action: + name: assign error + error parameter: 40.0 + - filter: Variable Assignment + assignments: + - name: GrossErrorProbability/totalSnowDepth + type: float + value: 0.02 + - name: BkgError/totalSnowDepth_background_error + type: float + value: 30.0 + - filter: Domain Check + where: + - variable: + name: ObsValue/totalSnowDepth + value: is_valid + actions: + - name: set + flag: missing_snowdepth + ignore: rejected observations + - name: reject + - filter: Domain Check + where: + - variable: + name: MetaData/stationElevation + value: is_valid + actions: + - name: set + flag: missing_elevation + ignore: rejected observations + - name: reject + - filter: Temporal Thinning + min_spacing: '{{ window_length }}' + seed_time: '{{ snow_background_time_iso }}' + category_variable: + name: MetaData/stationIdentification + actions: + - name: set + flag: temporal_thinning + ignore: rejected observations + - name: reject + obs prior filters: + - filter: Bounds Check + filter variables: + - name: totalSnowDepth + minvalue: 0.0 + maxvalue: 20000.0 + actions: + - name: set + flag: invalid_snowdepth + ignore: rejected observations + - name: reject + - filter: Domain Check # land only + where: + - variable: + name: GeoVaLs/slmsk + minvalue: 0.5 + maxvalue: 1.5 + actions: + - name: set + flag: land_check + ignore: rejected observations + - name: reject + - filter: Domain Check + where: + - variable: + name: MetaData/stationElevation + minvalue: -200.0 + maxvalue: 9900.0 + actions: + - name: set + flag: invalid_elevation + ignore: rejected observations + - name: reject + - filter: RejectList # no land-ice + where: + - variable: + name: GeoVaLs/vtype + minvalue: 14.5 + maxvalue: 15.5 + actions: + - name: set + flag: landice_check + ignore: rejected observations + - name: reject + - filter: Difference Check # elevation check + reference: MetaData/stationElevation + value: GeoVaLs/filtered_orography + threshold: 200. + actions: + - name: set + flag: elevation_bkgdiff + ignore: rejected observations + - name: reject + obs post filters: + - filter: Background Check # gross error check + filter variables: + - name: totalSnowDepth + threshold: 6.25 + actions: + - name: set + flag: background_check + ignore: rejected observations + - name: reject + - filter: Met Office Buddy Check + filter variables: + - name: totalSnowDepth + rejection_threshold: 0.5 + traced_boxes: # trace all observations + min_latitude: -90 + max_latitude: 90 + min_longitude: -180 + max_longitude: 180 + search_radius: 150 # km + station_id_variable: + name: MetaData/stationIdentification + num_zonal_bands: 24 + sort_by_pressure: false + max_total_num_buddies: 15 + max_num_buddies_from_single_band: 10 + max_num_buddies_with_same_station_id: 5 + use_legacy_buddy_collector: false + horizontal_correlation_scale: { "-90": 150, "90": 150 } + temporal_correlation_scale: PT6H + damping_factor_1: 1.0 + damping_factor_2: 1.0 + background_error_group: BkgError + actions: + - name: set + flag: buddy_check + ignore: rejected observations + - name: reject + diff --git a/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 b/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 index 827a8899f..d51d0d5ac 100644 --- a/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 @@ -7,7 +7,7 @@ obsdatain: engine: type: bufr - obsfile: "{{snow_obsdatain_path}}/{{snow_obsdatain_prefix}}sfcsno.tm00.bufr_d" + obsfile: "{{snow_obsdatain_path}}/{{snow_obsdatain_prefix}}{{observation_from_jcb}}{{snow_obsdatain_suffix}}" mapping file: "{{snow_obsdatain_path}}/bufr_sfcsno_mapping.yaml" missing file action: warn obsgrouping: @@ -15,7 +15,7 @@ obsdataout: engine: type: H5File - obsfile: "{{snow_obsdataout_path}}/{{snow_obsdataout_prefix}}sfcsno{{snow_obsdataout_suffix}}" + obsfile: "{{snow_obsdataout_path}}/{{snow_obsdataout_prefix}}{{observation_from_jcb}}{{snow_obsdataout_suffix}}" simulated variables: [totalSnowDepth] # diff --git a/parm/jcb-gdas/observations/snow/snocvr.yaml.j2 b/parm/jcb-gdas/observations/snow/snocvr.yaml.j2 new file mode 100644 index 000000000..c45029125 --- /dev/null +++ b/parm/jcb-gdas/observations/snow/snocvr.yaml.j2 @@ -0,0 +1,243 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: snocvr + obsdatain: + engine: + type: bufr + obsfile: "{{snow_obsdatain_path}}/{{snow_obsdatain_prefix}}{{observation_from_jcb}}{{snow_obsdatain_suffix}}" + mapping file: "{{snow_obsdatain_path}}/bufr_snocvr_mapping.yaml" + missing file action: warn + obsgrouping: + group variables: [stationIdentification] + obsdataout: + engine: + type: H5File + obsfile: "{{snow_obsdataout_path}}/{{snow_obsdataout_prefix}}{{observation_from_jcb}}{{snow_obsdataout_suffix}}" + simulated variables: [totalSnowDepth] + # + + # Observation Operator + # -------------------- + obs operator: + name: Composite + components: + - name: Identity + - name: BackgroundErrorIdentity + linear obs operator: + name: Identity + # + + # Observation Filters (QC) + # ------------------------ + obs pre filters: + - filter: Create Diagnostic Flags + flags: + - name: missing_snowdepth + initial value: false + - name: missing_elevation + initial value: false + - name: temporal_thinning + initial value: false + - name: invalid_snowdepth + initial value: false + - name: invalid_elevation + initial value: false + - name: land_check + initial value: false + - name: landice_check + initial value: false + - name: elevation_bkgdiff + initial value: false + - name: background_check + initial value: false + - name: buddy_check + initial value: false + - filter: Perform Action + filter variables: + - name: totalSnowDepth + action: + name: assign error + error parameter: 40.0 + - filter: Variable Assignment + assignments: + - name: GrossErrorProbability/totalSnowDepth + type: float + value: 0.02 + - name: BkgError/totalSnowDepth_background_error + type: float + value: 30.0 + - filter: Domain Check + where: + - variable: + name: ObsValue/totalSnowDepth + value: is_valid + actions: + - name: set + flag: missing_snowdepth + ignore: rejected observations + - name: reject + - filter: Domain Check + where: + - variable: + name: MetaData/stationElevation + value: is_valid + actions: + - name: set + flag: missing_elevation + ignore: rejected observations + - name: reject + - filter: Temporal Thinning + min_spacing: '{{ window_length }}' + seed_time: '{{ snow_background_time_iso }}' + category_variable: + name: MetaData/stationIdentification + actions: + - name: set + flag: temporal_thinning + ignore: rejected observations + - name: reject + obs prior filters: + - filter: Bounds Check + filter variables: + - name: totalSnowDepth + minvalue: 0.0 + maxvalue: 20000.0 + actions: + - name: set + flag: invalid_snowdepth + ignore: rejected observations + - name: reject + - filter: Domain Check # land only + where: + - variable: + name: GeoVaLs/slmsk + minvalue: 0.5 + maxvalue: 1.5 + actions: + - name: set + flag: land_check + ignore: rejected observations + - name: reject + - filter: Domain Check + where: + - variable: + name: MetaData/stationElevation + minvalue: -200.0 + maxvalue: 9900.0 + actions: + - name: set + flag: invalid_elevation + ignore: rejected observations + - name: reject + - filter: Domain Check # land only, no sea ice + where: + - variable: + name: GeoVaLs/fraction_of_ice + maxvalue: 0.0 + - filter: RejectList # no land-ice + where: + - variable: + name: GeoVaLs/vtype + minvalue: 14.5 + maxvalue: 15.5 + actions: + - name: set + flag: landice_check + ignore: rejected observations + - name: reject + - filter: Difference Check # elevation check + reference: MetaData/stationElevation + value: GeoVaLs/filtered_orography + threshold: 800. + actions: + - name: set + flag: elevation_bkgdiff + ignore: rejected observations + - name: reject + # Add inflate error characterized by a function of elevation difference + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/elevation_diff + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: [MetaData/stationElevation, GeoVaLs/filtered_orography] + coefficients: [1., -1.] + total exponent: 2 + total coefficient: 0.0000015625 #1/(800*800) + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/elevation_diff_exp + type: float + function: + name: ObsFunction/Exponential + options: + variables: [DerivedMetaData/elevation_diff] + coeffA: 1. + coeffB: -1. + coeffC: 0. + coeffD: 1. + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/elevation_diff_exp_inv + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: [DerivedMetaData/elevation_diff_exp] + total exponent: -1. + - filter: Perform Action + filter variables: + - name: totalSnowDepth + where: + - variable: + name: ObsValue/totalSnowDepth + minvalue: 0. + action: + name: inflate error + inflation variable: DerivedMetaData/elevation_diff_exp_inv # 2.0 + obs post filters: + - filter: Background Check # gross error check + filter variables: + - name: totalSnowDepth + threshold: 6.25 + absolute threshold: 250 + bias correction parameter: 1.0 + actions: + - name: set + flag: background_check + ignore: rejected observations + - name: reject + - filter: Met Office Buddy Check + filter variables: + - name: totalSnowDepth + rejection_threshold: 0.5 + traced_boxes: # trace all observations + min_latitude: -90 + max_latitude: 90 + min_longitude: -180 + max_longitude: 180 + search_radius: 150 # km + station_id_variable: + name: MetaData/stationIdentification + num_zonal_bands: 24 + sort_by_pressure: false + max_total_num_buddies: 15 + max_num_buddies_from_single_band: 10 + max_num_buddies_with_same_station_id: 5 + use_legacy_buddy_collector: false + horizontal_correlation_scale: { "-90": 150, "90": 150 } + temporal_correlation_scale: PT6H + damping_factor_1: 1.0 + damping_factor_2: 1.0 + background_error_group: BkgError + actions: + - name: set + flag: buddy_check + ignore: rejected observations + - name: reject + From 9ec7460bcacec38062ca00483f50eb23641f3d82 Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Thu, 8 May 2025 15:26:35 -0400 Subject: [PATCH 115/199] Update ims_snow_template.yaml.j2 (#120) Possible companion to https://github.com/NOAA-EMC/DA-utils/pull/25 --- parm/jcb-gdas/algorithm/obstats/snow/ims_snow_template.yaml.j2 | 3 +++ 1 file changed, 3 insertions(+) diff --git a/parm/jcb-gdas/algorithm/obstats/snow/ims_snow_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/snow/ims_snow_template.yaml.j2 index 059c5f348..012b7c15d 100644 --- a/parm/jcb-gdas/algorithm/obstats/snow/ims_snow_template.yaml.j2 +++ b/parm/jcb-gdas/algorithm/obstats/snow/ims_snow_template.yaml.j2 @@ -7,6 +7,9 @@ simulated variables: ['totalSnowDepth'] observed variables: ['totalSnowDepth'] statistics to compute: ['mean', 'count', 'RMS'] + variables: [totalSnowDepth] + groups to process: ['ombg', 'oman'] + qc groups: ['EffectiveQC0', 'EffectiveQC1'] output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_snow.nc" domains to process: - domain: From 9de83dbefc7fcdcca2786a928ba4f48b152e1ef5 Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Fri, 9 May 2025 19:25:58 +0000 Subject: [PATCH 116/199] Fix aero ioda-stats YAMLs (#121) The aerosol obs spaces were -180 to 180 for longitude, the ioda-stats code couldn't handle this, now this can be configured by a YAML option. --- .../obstats/aero/viirs_n20_template.yaml.j2 | 5 +- .../obstats/aero/viirs_n21_template.yaml.j2 | 49 +++++++++++++++++++ .../obstats/aero/viirs_npp_template.yaml.j2 | 5 +- 3 files changed, 57 insertions(+), 2 deletions(-) create mode 100644 parm/jcb-gdas/algorithm/obstats/aero/viirs_n21_template.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/obstats/aero/viirs_n20_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/aero/viirs_n20_template.yaml.j2 index 60bbbb2b2..b62a7c66b 100644 --- a/parm/jcb-gdas/algorithm/obstats/aero/viirs_n20_template.yaml.j2 +++ b/parm/jcb-gdas/algorithm/obstats/aero/viirs_n20_template.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: aod + name: aod_n20 obsdatain: engine: type: H5File @@ -11,6 +11,9 @@ qc groups: ['EffectiveQC0', 'EffectiveQC1'] statistics to compute: ['mean', 'count', 'RMS'] output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_aod.nc" + regular grid binning: + bin size in degrees: 5.0 + use negative longitudes: true domains to process: - domain: name: "SH" diff --git a/parm/jcb-gdas/algorithm/obstats/aero/viirs_n21_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/aero/viirs_n21_template.yaml.j2 new file mode 100644 index 000000000..e03a21981 --- /dev/null +++ b/parm/jcb-gdas/algorithm/obstats/aero/viirs_n21_template.yaml.j2 @@ -0,0 +1,49 @@ +- obs space: + name: aod_n21 + obsdatain: + engine: + type: H5File + obsfile: {{ aero_obsdatain_path }}/aero/diag_{{ obspace }}_aod_{{ stat_current_cycle_YMDH }}.nc + simulated variables: ['aerosolOpticalDepth'] + observed variables: ['aerosolOpticalDepth'] + variables: ['aerosolOpticalDepth'] + groups to process: ['ombg', 'oman'] + qc groups: ['EffectiveQC0', 'EffectiveQC1'] + statistics to compute: ['mean', 'count', 'RMS'] + output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_aod.nc" + regular grid binning: + bin size in degrees: 5.0 + use negative longitudes: true + domains to process: + - domain: + name: "SH" + first mask variable: latitude + first mask range: [-90,0] + - domain: + name: "NH" + first mask variable: latitude + first mask range: [0,90] + - domain: + name: "CONUS" + first mask variable: latitude + first mask range: [25,49] + second mask variable: longitude + second mask range: [-125,-66] + - domain: + name: "Europe" + first mask variable: latitude + first mask range: [35,70] + second mask variable: longitude + second mask range: [-11,38] + - domain: + name: "Africa" + first mask variable: latitude + first mask range: [-35,37] + second mask variable: longitude + second mask range: [-17,52] + - domain: + name: "Asia" + first mask variable: latitude + first mask range: [0,70] + second mask variable: longitude + second mask range: [38, 180] diff --git a/parm/jcb-gdas/algorithm/obstats/aero/viirs_npp_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/aero/viirs_npp_template.yaml.j2 index 60bbbb2b2..821769501 100644 --- a/parm/jcb-gdas/algorithm/obstats/aero/viirs_npp_template.yaml.j2 +++ b/parm/jcb-gdas/algorithm/obstats/aero/viirs_npp_template.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: aod + name: aod_npp obsdatain: engine: type: H5File @@ -11,6 +11,9 @@ qc groups: ['EffectiveQC0', 'EffectiveQC1'] statistics to compute: ['mean', 'count', 'RMS'] output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_aod.nc" + regular grid binning: + bin size in degrees: 5.0 + use negative longitudes: true domains to process: - domain: name: "SH" From 37f7ff9af5faae762148628d801bcc1d3060b174 Mon Sep 17 00:00:00 2001 From: DavidNew-NOAA <134300700+DavidNew-NOAA@users.noreply.github.com> Date: Thu, 15 May 2025 11:44:25 -0400 Subject: [PATCH 117/199] JCB templates for marine analysis refactor (#114) This PR, a companion to GDASApp PR #[1656](https://github.com/NOAA-EMC/GDASApp/pull/1656), creates and modifies JCB templates required for the marine analysis refactoring performed in Global Workflow PR [#3642](https://github.com/NOAA-EMC/global-workflow/pull/3642). This includes 1. Creation of templates for input YAMLs for various JEDI applications utilized in the marine analysis 2. Templates for obs staging using JCB (since JCB has access to a list of obs) 3. Refactoring of the soca obs stats utility input YAML to read a list of obs spaces formatted in a more standard way, that is, using JCB observation templates to build the input YAML. See Global Workflow Issue [#2931](https://github.com/NOAA-EMC/global-workflow/issues/2931) --- .../marine/soca_2cice_global.yaml.j2 | 20 +++++++++---------- .../algorithm/marine/soca_diag_stats.yaml.j2 | 17 ++++++++++++++++ .../marine/soca_diags_finalize.yaml.j2 | 7 +++++++ .../algorithm/marine/soca_obs_staging.yaml.j2 | 9 +++++++++ .../algorithm/marine/socaincr2mom6.yaml.j2 | 10 +++++----- 5 files changed, 48 insertions(+), 15 deletions(-) create mode 100644 parm/jcb-gdas/algorithm/marine/soca_diag_stats.yaml.j2 create mode 100644 parm/jcb-gdas/algorithm/marine/soca_diags_finalize.yaml.j2 create mode 100644 parm/jcb-gdas/algorithm/marine/soca_obs_staging.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/marine/soca_2cice_global.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_2cice_global.yaml.j2 index 4ccf033cf..25be1a7aa 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_2cice_global.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_2cice_global.yaml.j2 @@ -23,23 +23,23 @@ variable change: min hice: 0.5 min hsno: 0.1 cice background state: - restart: {{ ice_rst }} + restart: ./Data/{{ marine_cice_date_fms }}.cice_model.res.nc ncat: 5 ice_lev: 7 sno_lev: 1 cice output: - restart: {{ ice_rst }} + restart: ./Data/{{ marine_cice_date_fms }}.cice_model.res.nc increment output: datadir: Data exp: soca2cice type: incr - date: '{{ fcst_begin }}' + date: {{ marine_cice_date_iso }} soca increment: read_from_file: 1 basename: ./ - ocn_filename: {{ ocn_inc }} - ice_filename: {{ ice_inc }} - date: '{{ fcst_begin }}' + ocn_filename: ./Data/ocn.3dvarfgat_pseudo.incr.{{ marine_window_middle_iso }}.nc + ice_filename: ./Data/ice.3dvarfgat_pseudo.incr.{{ marine_window_middle_iso }}.nc + date: {{ marine_cice_date_iso }} output variables: - sea_water_potential_temperature - sea_water_salinity @@ -51,9 +51,9 @@ states: - input: read_from_file: 1 basename: ./ - ocn_filename: {{ ocn_ana }} - ice_filename: {{ ice_ana }} - date: '{{ fcst_begin }}' + ocn_filename: ./Data/ocn.3dvarfgat_pseudo.an.{{ marine_window_middle_iso }}.nc + ice_filename: ./Data/ice.3dvarfgat_pseudo.an.{{ marine_window_middle_iso }}.nc + date: {{ marine_cice_date_iso }} state variables: - sea_water_potential_temperature - sea_water_salinity @@ -65,4 +65,4 @@ states: datadir: Data exp: soca2cice type: fc - date: '{{ fcst_begin }}' + date: {{ marine_cice_date_iso }} diff --git a/parm/jcb-gdas/algorithm/marine/soca_diag_stats.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_diag_stats.yaml.j2 new file mode 100644 index 000000000..020103d1f --- /dev/null +++ b/parm/jcb-gdas/algorithm/marine/soca_diag_stats.yaml.j2 @@ -0,0 +1,17 @@ +time window: + begin: '1900-01-01T00:00:00Z' + end: '2035-01-01T00:00:00Z' + bound to include: begin +observations: + observers: +{% for observation_from_jcb in observations %} +{% if use_observer(observation_from_jcb) %} +{% filter indent(width=4) %} +{% include observation_from_jcb + '.yaml.j2' %} +{% filter indent(width=2) %} +experiment identifier: '{{ pslot }}' +csv output: '{{marine_obsdataout_path}}/{{ marine_obsdatain_prefix }}ocn.{{ observation_from_jcb }}.stats.csv' +{% endfilter %} +{% endfilter %} +{% endif %} +{% endfor %} diff --git a/parm/jcb-gdas/algorithm/marine/soca_diags_finalize.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_diags_finalize.yaml.j2 new file mode 100644 index 000000000..3f1f1e3f6 --- /dev/null +++ b/parm/jcb-gdas/algorithm/marine/soca_diags_finalize.yaml.j2 @@ -0,0 +1,7 @@ +copy_opt: +{% for observation_from_jcb in observations %} +{% if use_observer(observation_from_jcb) %} +- ['{{marine_obsdataout_path}}/{{observation_from_jcb}}{{marine_obsdataout_suffix}}', '{{marine_obsdatastats_path}}'] +- ['{{marine_obsdataout_path}}/{{marine_obsdatain_prefix}}ocn.{{observation_from_jcb}}.stats.csv', '{{marine_obsdatastats_path}}'] +{% endif %} +{% endfor %} diff --git a/parm/jcb-gdas/algorithm/marine/soca_obs_staging.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_obs_staging.yaml.j2 new file mode 100644 index 000000000..49b972bd1 --- /dev/null +++ b/parm/jcb-gdas/algorithm/marine/soca_obs_staging.yaml.j2 @@ -0,0 +1,9 @@ +mkdir: + - '{{marine_obsdatain_path}}' + - '{{marine_obsdataout_path}}' +copy_opt: + {% for observation_from_jcb in observations %} + {% if use_observer(observation_from_jcb) %} + - ['{{marine_obsdataroot_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}', '{{marine_obsdatain_path}}'] + {% endif %} + {% endfor %} diff --git a/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 b/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 index 85b8f0396..4218766be 100644 --- a/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 @@ -2,7 +2,7 @@ geometry: mom6_input_nml: mom_input.nml fields metadata: ./fields_metadata.yaml -date: '{{ marine_window_begin }}' +date: '{{ marine_window_begin_iso }}' layers variable: [sea_water_cell_thickness] @@ -20,20 +20,20 @@ set increment variables to zero: - northward_sea_water_velocity vertical geometry: - date: '{{ marine_window_begin }}' + date: '{{ marine_window_begin_iso }}' basename: ./INPUT/ ocn_filename: MOM.res.nc read_from_file: 1 soca increment: - date: '{{ marine_window_begin }}' + date: '{{ marine_window_begin_iso }}' basename: ./Data/ - ocn_filename: 'ocn.3dvarfgat_pseudo.incr.{{marine_window_middle}}.nc' + ocn_filename: 'ocn.3dvarfgat_pseudo.incr.{{ marine_window_middle_iso }}.nc' read_from_file: 1 output increment: datadir: ./ - date: '{{ marine_window_begin }}' + date: '{{ marine_window_begin_iso }}' exp: mom6_iau type: incr output file: inc.nc From 1a30a73686a263ca24d5779f44d408a522299b81 Mon Sep 17 00:00:00 2001 From: yuanxue2870 <136842224+yuanxue2870@users.noreply.github.com> Date: Mon, 19 May 2025 10:15:53 -0400 Subject: [PATCH 118/199] Revise IMS ioda-stats YAML and add other snow stats YAMLs (#123) This PR directly solves Issue: https://github.com/NOAA-EMC/jcb-gdas/issues/122 Edits made in this PR include: 1) fix a bug in the current IMS ioda-stats YAML since diag_ims file uses longitude from 0 to 360, instead of -180 to 180. 2) add `snocvr_snow.yaml` and `sfcsno.yaml` for outputing ioda-stats. 3) add other domains of interest to all three snow-related YAMLs per discussions on 05/15/2025 land team meeting. No significant changes of the output are expected as a result of this PR. Note: the bin size of the ims_snow.yaml is set to 2 deg for now due to run time consideration. 1 deg bin takes a much longer time for ioda-stats.exe to process diag_ims file at a relatively fine resolution (e.g., C384). A G-W CI test of `C96C48mx500_S2SW_cyc_gfs` was conducted: the concatenated `snow.yaml` looks OK. A follow-up PR will be submitted to G-W to make sure snocvr stats (and snocvr.yaml) and sfcsno stats (and sfcsno.yaml) can be output as well. --- .../obstats/snow/ims_snow_template.yaml.j2 | 30 ++++++-- .../obstats/snow/sfcsno_template.yaml.j2 | 68 +++++++++++++++++++ .../obstats/snow/snocvr_snow_template.yaml.j2 | 68 +++++++++++++++++++ 3 files changed, 162 insertions(+), 4 deletions(-) create mode 100644 parm/jcb-gdas/algorithm/obstats/snow/sfcsno_template.yaml.j2 create mode 100644 parm/jcb-gdas/algorithm/obstats/snow/snocvr_snow_template.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/obstats/snow/ims_snow_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/snow/ims_snow_template.yaml.j2 index 012b7c15d..cfc9898e0 100644 --- a/parm/jcb-gdas/algorithm/obstats/snow/ims_snow_template.yaml.j2 +++ b/parm/jcb-gdas/algorithm/obstats/snow/ims_snow_template.yaml.j2 @@ -11,6 +11,9 @@ groups to process: ['ombg', 'oman'] qc groups: ['EffectiveQC0', 'EffectiveQC1'] output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_snow.nc" + regular grid binning: + bin size in degrees: 2.0 + use negative longitudes: false domains to process: - domain: name: "SH" @@ -21,26 +24,45 @@ first mask variable: latitude first mask range: [0,90] - domain: - name: "CONUS" + name: "West_CONUS" first mask variable: latitude first mask range: [25,49] second mask variable: longitude - second mask range: [-125,-66] + second mask range: [235,260] + - domain: + name: "East_CONUS" + first mask variable: latitude + first mask range: [25,49] + second mask variable: longitude + second mask range: [260,294] + - domain: + name: "Alaska" + first mask variable: latitude + first mask range: [52,72] + second mask variable: longitude + second mask range: [190,230] - domain: name: "Europe" first mask variable: latitude first mask range: [35,70] second mask variable: longitude - second mask range: [-11,38] + second mask range: [38,349] - domain: name: "Africa" first mask variable: latitude first mask range: [-35,37] second mask variable: longitude - second mask range: [-17,52] + second mask range: [52,343] - domain: name: "Asia" first mask variable: latitude first mask range: [0,70] second mask variable: longitude second mask range: [38, 180] + - domain: + name: "High_Mount_Asia" + first mask variable: latitude + first mask range: [20,46] + second mask variable: longitude + second mask range: [60, 111] + diff --git a/parm/jcb-gdas/algorithm/obstats/snow/sfcsno_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/snow/sfcsno_template.yaml.j2 new file mode 100644 index 000000000..08f5a03ab --- /dev/null +++ b/parm/jcb-gdas/algorithm/obstats/snow/sfcsno_template.yaml.j2 @@ -0,0 +1,68 @@ +- obs space: + name: sfcsno + obsdatain: + engine: + type: H5File + obsfile: {{ snow_obsdatain_path }}/snow/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc + simulated variables: ['totalSnowDepth'] + observed variables: ['totalSnowDepth'] + statistics to compute: ['mean', 'count', 'RMS'] + variables: [totalSnowDepth] + groups to process: ['ombg', 'oman'] + qc groups: ['EffectiveQC0', 'EffectiveQC1'] + output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_snow.nc" + regular grid binning: + bin size in degrees: 1.0 + use negative longitudes: true + domains to process: + - domain: + name: "SH" + first mask variable: latitude + first mask range: [-90,0] + - domain: + name: "NH" + first mask variable: latitude + first mask range: [0,90] + - domain: + name: "West_CONUS" + first mask variable: latitude + first mask range: [25,49] + second mask variable: longitude + second mask range: [-125,-100] + - domain: + name: "East_CONUS" + first mask variable: latitude + first mask range: [25,49] + second mask variable: longitude + second mask range: [-100,-66] + - domain: + name: "Alaska" + first mask variable: latitude + first mask range: [52,72] + second mask variable: longitude + second mask range: [-170,-130] + - domain: + name: "Europe" + first mask variable: latitude + first mask range: [35,70] + second mask variable: longitude + second mask range: [-11,38] + - domain: + name: "Africa" + first mask variable: latitude + first mask range: [-35,37] + second mask variable: longitude + second mask range: [-17,52] + - domain: + name: "Asia" + first mask variable: latitude + first mask range: [0,70] + second mask variable: longitude + second mask range: [38, 180] + - domain: + name: "High_Mount_Asia" + first mask variable: latitude + first mask range: [20,46] + second mask variable: longitude + second mask range: [60, 111] + diff --git a/parm/jcb-gdas/algorithm/obstats/snow/snocvr_snow_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/snow/snocvr_snow_template.yaml.j2 new file mode 100644 index 000000000..8f23b0162 --- /dev/null +++ b/parm/jcb-gdas/algorithm/obstats/snow/snocvr_snow_template.yaml.j2 @@ -0,0 +1,68 @@ +- obs space: + name: snocvr_snow + obsdatain: + engine: + type: H5File + obsfile: {{ snow_obsdatain_path }}/snow/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc + simulated variables: ['totalSnowDepth'] + observed variables: ['totalSnowDepth'] + statistics to compute: ['mean', 'count', 'RMS'] + variables: [totalSnowDepth] + groups to process: ['ombg', 'oman'] + qc groups: ['EffectiveQC0', 'EffectiveQC1'] + output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_snow.nc" + regular grid binning: + bin size in degrees: 1.0 + use negative longitudes: true + domains to process: + - domain: + name: "SH" + first mask variable: latitude + first mask range: [-90,0] + - domain: + name: "NH" + first mask variable: latitude + first mask range: [0,90] + - domain: + name: "West_CONUS" + first mask variable: latitude + first mask range: [25,49] + second mask variable: longitude + second mask range: [-125,-100] + - domain: + name: "East_CONUS" + first mask variable: latitude + first mask range: [25,49] + second mask variable: longitude + second mask range: [-100,-66] + - domain: + name: "Alaska" + first mask variable: latitude + first mask range: [52,72] + second mask variable: longitude + second mask range: [-170,-130] + - domain: + name: "Europe" + first mask variable: latitude + first mask range: [35,70] + second mask variable: longitude + second mask range: [-11,38] + - domain: + name: "Africa" + first mask variable: latitude + first mask range: [-35,37] + second mask variable: longitude + second mask range: [-17,52] + - domain: + name: "Asia" + first mask variable: latitude + first mask range: [0,70] + second mask variable: longitude + second mask range: [38, 180] + - domain: + name: "High_Mount_Asia" + first mask variable: latitude + first mask range: [20,46] + second mask variable: longitude + second mask range: [60, 111] + From a1c84909abc13f619e66d1cb7fe6b7702711d541 Mon Sep 17 00:00:00 2001 From: yuanxue2870 <136842224+yuanxue2870@users.noreply.github.com> Date: Tue, 20 May 2025 11:56:12 -0400 Subject: [PATCH 119/199] rename "snocvr_snow" to "snocvr" (#125) This PR directly addresses the Issue: https://github.com/NOAA-EMC/jcb-gdas/issues/124. In all, (outdated) "snocvr_snow" = (now) "madis_snow" for use for 2021-2023 observations. "snocvr" is used to denote 2023 to current snocvr observations. Based on conversations with Jiarui, several draft PRs in G-W and GDAS will be submitted to handle the switch among different snocvr names automatically. Again, thanks for discussions with Jiarui Dong @jiaruidong2017 for the clarification of the issue! --- ...te.yaml.j2 => madis_snow_template.yaml.j2} | 2 +- .../obstats/snow/snocvr_template.yaml.j2 | 68 +++++++++++++++++++ 2 files changed, 69 insertions(+), 1 deletion(-) rename parm/jcb-gdas/algorithm/obstats/snow/{snocvr_snow_template.yaml.j2 => madis_snow_template.yaml.j2} (98%) create mode 100644 parm/jcb-gdas/algorithm/obstats/snow/snocvr_template.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/obstats/snow/snocvr_snow_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/snow/madis_snow_template.yaml.j2 similarity index 98% rename from parm/jcb-gdas/algorithm/obstats/snow/snocvr_snow_template.yaml.j2 rename to parm/jcb-gdas/algorithm/obstats/snow/madis_snow_template.yaml.j2 index 8f23b0162..e67a9952d 100644 --- a/parm/jcb-gdas/algorithm/obstats/snow/snocvr_snow_template.yaml.j2 +++ b/parm/jcb-gdas/algorithm/obstats/snow/madis_snow_template.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: snocvr_snow + name: madis_snow obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/algorithm/obstats/snow/snocvr_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/snow/snocvr_template.yaml.j2 new file mode 100644 index 000000000..47a7a1e08 --- /dev/null +++ b/parm/jcb-gdas/algorithm/obstats/snow/snocvr_template.yaml.j2 @@ -0,0 +1,68 @@ +- obs space: + name: snocvr + obsdatain: + engine: + type: H5File + obsfile: {{ snow_obsdatain_path }}/snow/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc + simulated variables: ['totalSnowDepth'] + observed variables: ['totalSnowDepth'] + statistics to compute: ['mean', 'count', 'RMS'] + variables: [totalSnowDepth] + groups to process: ['ombg', 'oman'] + qc groups: ['EffectiveQC0', 'EffectiveQC1'] + output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_snow.nc" + regular grid binning: + bin size in degrees: 1.0 + use negative longitudes: true + domains to process: + - domain: + name: "SH" + first mask variable: latitude + first mask range: [-90,0] + - domain: + name: "NH" + first mask variable: latitude + first mask range: [0,90] + - domain: + name: "West_CONUS" + first mask variable: latitude + first mask range: [25,49] + second mask variable: longitude + second mask range: [-125,-100] + - domain: + name: "East_CONUS" + first mask variable: latitude + first mask range: [25,49] + second mask variable: longitude + second mask range: [-100,-66] + - domain: + name: "Alaska" + first mask variable: latitude + first mask range: [52,72] + second mask variable: longitude + second mask range: [-170,-130] + - domain: + name: "Europe" + first mask variable: latitude + first mask range: [35,70] + second mask variable: longitude + second mask range: [-11,38] + - domain: + name: "Africa" + first mask variable: latitude + first mask range: [-35,37] + second mask variable: longitude + second mask range: [-17,52] + - domain: + name: "Asia" + first mask variable: latitude + first mask range: [0,70] + second mask variable: longitude + second mask range: [38, 180] + - domain: + name: "High_Mount_Asia" + first mask variable: latitude + first mask range: [20,46] + second mask variable: longitude + second mask range: [60, 111] + From 10b70d3c9a30e0ac0294e68cfee485f6e21bcd9f Mon Sep 17 00:00:00 2001 From: DavidNew-NOAA <134300700+DavidNew-NOAA@users.noreply.github.com> Date: Wed, 21 May 2025 20:28:27 -0400 Subject: [PATCH 120/199] Marine refactor revisions (#126) This PR makes a new algorithm template for the soca_diag_stats application, based on debugging of my previous revisions in https://github.com/NOAA-EMC/global-workflow/pull/3642 Companion: https://github.com/NOAA-EMC/GDASApp/pull/1707 --- .../algorithm/marine/soca_diag_stats.yaml.j2 | 24 +++++++++---------- 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/parm/jcb-gdas/algorithm/marine/soca_diag_stats.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_diag_stats.yaml.j2 index 020103d1f..cc06fa645 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_diag_stats.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_diag_stats.yaml.j2 @@ -2,16 +2,16 @@ time window: begin: '1900-01-01T00:00:00Z' end: '2035-01-01T00:00:00Z' bound to include: begin -observations: - observers: -{% for observation_from_jcb in observations %} -{% if use_observer(observation_from_jcb) %} -{% filter indent(width=4) %} -{% include observation_from_jcb + '.yaml.j2' %} -{% filter indent(width=2) %} -experiment identifier: '{{ pslot }}' -csv output: '{{marine_obsdataout_path}}/{{ marine_obsdatain_prefix }}ocn.{{ observation_from_jcb }}.stats.csv' -{% endfilter %} -{% endfilter %} -{% endif %} +obs spaces: +{% for observation_from_jcb in cleaned_observations %} +- obs space: + name: '{{ observation_from_jcb }}' + obsdatain: + engine: + type: H5File + obsfile: "{{ marine_obsdataout_path }}/{{ marine_obsdataout_prefix }}{{ observation_from_jcb }}{{ marine_obsdataout_suffix }}" + simulated variables: ['{{ obs_variables[observation_from_jcb] }}'] + variable: '{{ obs_variables[observation_from_jcb] }}' + experiment identifier: '{{ pslot }}' + csv output: '{{marine_obsdataout_path}}/{{ marine_obsdatain_prefix }}ocn.{{ observation_from_jcb }}.stats.csv' {% endfor %} From 690e46491614c63da5eaab45e0efdb360c7fbb74 Mon Sep 17 00:00:00 2001 From: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com> Date: Tue, 27 May 2025 08:56:17 -0400 Subject: [PATCH 121/199] Refactor the FieldMetadata and disable variable name aliasing (#98) --- .../aero/aero_convert_background.yaml.j2 | 59 ++++++-- .../aero/aero_gen_bmatrix_diagb.yaml.j2 | 94 +++++++++++-- .../aero/aero_gen_bmatrix_diffusion.yaml.j2 | 42 ++++-- .../atmosphere/atm_addincrement.yaml.j2 | 50 ++++++- .../fv3jedi_correction_increment.yaml.j2 | 45 +++++- .../fv3jedi_ensemble_recenter.yaml.j2 | 54 +++++++- .../atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 | 64 +++++++-- .../fv3jedi_fv3inc_variational.yaml.j2 | 61 +++++++-- .../snow/fv3jedi_land_ensrecenter.yaml.j2 | 27 +++- .../snow/fv3jedi_snow_ensmean.yaml.j2 | 25 +++- .../aero/aero_3dvar_outer_loop_1.yaml.j2 | 1 - .../aero/aero_3dvar_outer_loop_2.yaml.j2 | 1 - .../model/aero/aero_background.yaml.j2 | 41 +++++- ..._background_error_static_diffusion.yaml.j2 | 49 +++++-- .../aero_final_increment_cubed_sphere.yaml.j2 | 40 +++++- .../aero_final_increment_gaussian.yaml.j2 | 25 ++-- .../aero/aero_geometry_background.yaml.j2 | 1 - .../model/aero/aero_model_pseudo.yaml.j2 | 41 +++++- .../atmosphere_3dvar_outer_loop_1.yaml.j2 | 1 - .../atmosphere_3dvar_outer_loop_2.yaml.j2 | 1 - .../atmosphere/atmosphere_background.yaml.j2 | 62 ++++++++- .../atmosphere_background_ensemble.yaml.j2 | 63 ++++++++- ..._background_error_hybrid_bump_bump.yaml.j2 | 128 ------------------ ...ackground_error_hybrid_gsibec_bump.yaml.j2 | 79 +++++++++-- ...phere_background_error_static_bump.yaml.j2 | 89 ------------ ...ere_background_error_static_gsibec.yaml.j2 | 22 ++- ...phere_final_increment_cubed_sphere.yaml.j2 | 20 ++- ...tmosphere_final_increment_gaussian.yaml.j2 | 10 +- .../atmosphere_geometry_background.yaml.j2 | 1 - .../atmosphere_model_pseudo.yaml.j2 | 25 +++- ...t_ensemble_increments_cubed_sphere.yaml.j2 | 9 ++ ...utput_ensemble_increments_gaussian.yaml.j2 | 9 ++ ...mosphere_posterior_output_gaussian.yaml.j2 | 9 ++ .../snow/snow_3dvar_outer_loop_1.yaml.j2 | 6 +- .../model/snow/snow_background.yaml.j2 | 12 +- .../snow/snow_final_increment_fms.yaml.j2 | 8 +- .../snow/snow_geometry_background.yaml.j2 | 1 - .../gdas-atmosphere-templates.yaml | 2 +- 38 files changed, 907 insertions(+), 370 deletions(-) delete mode 100644 parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_bump_bump.yaml.j2 delete mode 100644 parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_bump.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/aero/aero_convert_background.yaml.j2 b/parm/jcb-gdas/algorithm/aero/aero_convert_background.yaml.j2 index 0a1344fb7..d3f570b87 100644 --- a/parm/jcb-gdas/algorithm/aero/aero_convert_background.yaml.j2 +++ b/parm/jcb-gdas/algorithm/aero/aero_convert_background.yaml.j2 @@ -9,7 +9,6 @@ input geometry: npx: {{ aero_npx_ges }} npy: {{ aero_npy_ges }} npz: {{ aero_npz_ges }} - field metadata override: "{{aero_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" output geometry: fms initialization: namelist filename: "{{aero_fv3jedi_files_path}}/fmsmpp.nml" @@ -21,21 +20,41 @@ output geometry: npx: {{ aero_npx_anl }} npy: {{ aero_npy_anl }} npz: {{ aero_npz_anl }} - field metadata override: "{{aero_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" states: - input: datetime: '{{ aero_background_error_time_iso }}' filetype: fms restart - state variables: [mass_fraction_of_sulfate_in_air, - mass_fraction_of_hydrophobic_black_carbon_in_air, - mass_fraction_of_hydrophilic_black_carbon_in_air, - mass_fraction_of_hydrophobic_organic_carbon_in_air, - mass_fraction_of_hydrophilic_organic_carbon_in_air, - mass_fraction_of_dust001_in_air, mass_fraction_of_dust002_in_air, - mass_fraction_of_dust003_in_air, mass_fraction_of_dust004_in_air, - mass_fraction_of_dust005_in_air, mass_fraction_of_sea_salt001_in_air, - mass_fraction_of_sea_salt002_in_air, mass_fraction_of_sea_salt003_in_air, - mass_fraction_of_sea_salt004_in_air] + state variables: + - mass_fraction_of_sulfate_in_air + - mass_fraction_of_hydrophobic_black_carbon_in_air + - mass_fraction_of_hydrophilic_black_carbon_in_air + - mass_fraction_of_hydrophobic_organic_carbon_in_air + - mass_fraction_of_hydrophilic_organic_carbon_in_air + - mass_fraction_of_dust001_in_air + - mass_fraction_of_dust002_in_air + - mass_fraction_of_dust003_in_air + - mass_fraction_of_dust004_in_air + - mass_fraction_of_dust005_in_air + - mass_fraction_of_sea_salt001_in_air + - mass_fraction_of_sea_salt002_in_air + - mass_fraction_of_sea_salt003_in_air + - mass_fraction_of_sea_salt004_in_air + field io names: + mass_fraction_of_dust001_in_air: dust1 + mass_fraction_of_dust002_in_air: dust2 + mass_fraction_of_dust003_in_air: dust3 + mass_fraction_of_dust004_in_air: dust4 + mass_fraction_of_dust005_in_air: dust5 + mass_fraction_of_sea_salt001_in_air: seas1 + mass_fraction_of_sea_salt002_in_air: seas2 + mass_fraction_of_sea_salt003_in_air: seas3 + mass_fraction_of_sea_salt004_in_air: seas4 + mass_fraction_of_sea_salt005_in_air: seas5 + mass_fraction_of_hydrophobic_black_carbon_in_air: bc1 + mass_fraction_of_hydrophilic_black_carbon_in_air: bc2 + mass_fraction_of_hydrophobic_organic_carbon_in_air: oc1 + mass_fraction_of_hydrophilic_organic_carbon_in_air: oc2 + mass_fraction_of_sulfate_in_air: so4 datapath: {{ aero_background_path }} filename_core: '{{ aero_background_error_time_fv3 }}.fv_core.res.nc' filename_trcr: '{{ aero_background_error_time_fv3 }}.fv_tracer.res.nc' @@ -46,3 +65,19 @@ states: filename_core: 'anlres.fv_core.res.nc' filename_trcr: 'anlres.fv_tracer.res.nc' filename_cplr: 'anlres.coupler.res' + field io names: + mass_fraction_of_dust001_in_air: dust1 + mass_fraction_of_dust002_in_air: dust2 + mass_fraction_of_dust003_in_air: dust3 + mass_fraction_of_dust004_in_air: dust4 + mass_fraction_of_dust005_in_air: dust5 + mass_fraction_of_sea_salt001_in_air: seas1 + mass_fraction_of_sea_salt002_in_air: seas2 + mass_fraction_of_sea_salt003_in_air: seas3 + mass_fraction_of_sea_salt004_in_air: seas4 + mass_fraction_of_sea_salt005_in_air: seas5 + mass_fraction_of_hydrophobic_black_carbon_in_air: bc1 + mass_fraction_of_hydrophilic_black_carbon_in_air: bc2 + mass_fraction_of_hydrophobic_organic_carbon_in_air: oc1 + mass_fraction_of_hydrophilic_organic_carbon_in_air: oc2 + mass_fraction_of_sulfate_in_air: so4 diff --git a/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diagb.yaml.j2 b/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diagb.yaml.j2 index d90756eb3..05b1fb637 100644 --- a/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diagb.yaml.j2 +++ b/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diagb.yaml.j2 @@ -9,21 +9,41 @@ geometry: npx: {{ aero_npx_anl }} npy: {{ aero_npy_anl }} npz: {{ aero_npz_anl }} - field metadata override: "{{aero_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" date: '{{ aero_background_error_time_iso }}' background: datetime: '{{ aero_background_error_time_iso }}' filetype: fms restart - state variables: [mass_fraction_of_sulfate_in_air, - mass_fraction_of_hydrophobic_black_carbon_in_air, - mass_fraction_of_hydrophilic_black_carbon_in_air, - mass_fraction_of_hydrophobic_organic_carbon_in_air, - mass_fraction_of_hydrophilic_organic_carbon_in_air, - mass_fraction_of_dust001_in_air, mass_fraction_of_dust002_in_air, - mass_fraction_of_dust003_in_air, mass_fraction_of_dust004_in_air, - mass_fraction_of_dust005_in_air, mass_fraction_of_sea_salt001_in_air, - mass_fraction_of_sea_salt002_in_air, mass_fraction_of_sea_salt003_in_air, - mass_fraction_of_sea_salt004_in_air] + state variables: + - mass_fraction_of_sulfate_in_air + - mass_fraction_of_hydrophobic_black_carbon_in_air + - mass_fraction_of_hydrophilic_black_carbon_in_air + - mass_fraction_of_hydrophobic_organic_carbon_in_air + - mass_fraction_of_hydrophilic_organic_carbon_in_air + - mass_fraction_of_dust001_in_air + - mass_fraction_of_dust002_in_air + - mass_fraction_of_dust003_in_air + - mass_fraction_of_dust004_in_air + - mass_fraction_of_dust005_in_air + - mass_fraction_of_sea_salt001_in_air + - mass_fraction_of_sea_salt002_in_air + - mass_fraction_of_sea_salt003_in_air + - mass_fraction_of_sea_salt004_in_air + field io names: + mass_fraction_of_dust001_in_air: dust1 + mass_fraction_of_dust002_in_air: dust2 + mass_fraction_of_dust003_in_air: dust3 + mass_fraction_of_dust004_in_air: dust4 + mass_fraction_of_dust005_in_air: dust5 + mass_fraction_of_sea_salt001_in_air: seas1 + mass_fraction_of_sea_salt002_in_air: seas2 + mass_fraction_of_sea_salt003_in_air: seas3 + mass_fraction_of_sea_salt004_in_air: seas4 + mass_fraction_of_sea_salt005_in_air: seas5 + mass_fraction_of_hydrophobic_black_carbon_in_air: bc1 + mass_fraction_of_hydrophilic_black_carbon_in_air: bc2 + mass_fraction_of_hydrophobic_organic_carbon_in_air: oc1 + mass_fraction_of_hydrophilic_organic_carbon_in_air: oc2 + mass_fraction_of_sulfate_in_air: so4 datapath: {{ aero_background_path }} filename_core: '{{ aero_background_error_time_fv3 }}.anlres.fv_core.res.nc' filename_trcr: '{{ aero_background_error_time_fv3 }}.anlres.fv_tracer.res.nc' @@ -33,7 +53,22 @@ background error: datapath: {{ aero_standard_deviation_path }} filename_trcr: stddev.fv_tracer.res.nc filename_cplr: stddev.coupler.res - + field io names: + mass_fraction_of_dust001_in_air: dust1 + mass_fraction_of_dust002_in_air: dust2 + mass_fraction_of_dust003_in_air: dust3 + mass_fraction_of_dust004_in_air: dust4 + mass_fraction_of_dust005_in_air: dust5 + mass_fraction_of_sea_salt001_in_air: seas1 + mass_fraction_of_sea_salt002_in_air: seas2 + mass_fraction_of_sea_salt003_in_air: seas3 + mass_fraction_of_sea_salt004_in_air: seas4 + mass_fraction_of_sea_salt005_in_air: seas5 + mass_fraction_of_hydrophobic_black_carbon_in_air: bc1 + mass_fraction_of_hydrophilic_black_carbon_in_air: bc2 + mass_fraction_of_hydrophobic_organic_carbon_in_air: oc1 + mass_fraction_of_hydrophilic_organic_carbon_in_air: oc2 + mass_fraction_of_sulfate_in_air: so4 climate background error: geometry: fms initialization: @@ -46,13 +81,28 @@ climate background error: npx: {{ aero_npx_clim_b }} npy: {{ aero_npy_clim_b }} npz: {{ aero_npz_clim_b }} - field metadata override: "{{aero_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" climate background error stddev: filetype: fms restart skip coupler file: true datapath: {{ aero_climatological_b_path }} filename_trcr: stddev.fv_tracer.res.nc filename_cplr: stddev.coupler.res + field io names: + mass_fraction_of_dust001_in_air: dust1 + mass_fraction_of_dust002_in_air: dust2 + mass_fraction_of_dust003_in_air: dust3 + mass_fraction_of_dust004_in_air: dust4 + mass_fraction_of_dust005_in_air: dust5 + mass_fraction_of_sea_salt001_in_air: seas1 + mass_fraction_of_sea_salt002_in_air: seas2 + mass_fraction_of_sea_salt003_in_air: seas3 + mass_fraction_of_sea_salt004_in_air: seas4 + mass_fraction_of_sea_salt005_in_air: seas5 + mass_fraction_of_hydrophobic_black_carbon_in_air: bc1 + mass_fraction_of_hydrophilic_black_carbon_in_air: bc2 + mass_fraction_of_hydrophobic_organic_carbon_in_air: oc1 + mass_fraction_of_hydrophilic_organic_carbon_in_air: oc2 + mass_fraction_of_sulfate_in_air: so4 diagb weight: {{ aero_diagb_weight }} staticb rescaling factor: {{ aero_diagb_static_rescale_factor }} @@ -85,14 +135,28 @@ global rescale: npx: {{ aero_npx_clim_b }} npy: {{ aero_npy_clim_b }} npz: {{ aero_npz_clim_b }} - field metadata override: "{{aero_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" rescale stddev: filetype: fms restart skip coupler file: true datapath: {{ aero_rescale_b_path }} filename_trcr: rescale.fv_tracer.res.nc filename_cplr: rescale.coupler.res - + field io names: + mass_fraction_of_dust001_in_air: dust1 + mass_fraction_of_dust002_in_air: dust2 + mass_fraction_of_dust003_in_air: dust3 + mass_fraction_of_dust004_in_air: dust4 + mass_fraction_of_dust005_in_air: dust5 + mass_fraction_of_sea_salt001_in_air: seas1 + mass_fraction_of_sea_salt002_in_air: seas2 + mass_fraction_of_sea_salt003_in_air: seas3 + mass_fraction_of_sea_salt004_in_air: seas4 + mass_fraction_of_sea_salt005_in_air: seas5 + mass_fraction_of_hydrophobic_black_carbon_in_air: bc1 + mass_fraction_of_hydrophilic_black_carbon_in_air: bc2 + mass_fraction_of_hydrophobic_organic_carbon_in_air: oc1 + mass_fraction_of_hydrophilic_organic_carbon_in_air: oc2 + mass_fraction_of_sulfate_in_air: so4 number of halo points: {{ aero_diagb_n_halo }} number of neighbors: {{ aero_diagb_n_neighbors }} simple smoothing: diff --git a/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diffusion.yaml.j2 b/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diffusion.yaml.j2 index 0462daaca..9742aaf90 100644 --- a/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diffusion.yaml.j2 +++ b/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diffusion.yaml.j2 @@ -9,21 +9,41 @@ geometry: npx: {{ aero_npx_anl }} npy: {{ aero_npy_anl }} npz: {{ aero_npz_anl }} - field metadata override: "{{aero_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" date: '{{ aero_background_error_time_iso }}' background: datetime: '{{ aero_background_error_time_iso }}' filetype: fms restart - state variables: [mass_fraction_of_sulfate_in_air, - mass_fraction_of_hydrophobic_black_carbon_in_air, - mass_fraction_of_hydrophilic_black_carbon_in_air, - mass_fraction_of_hydrophobic_organic_carbon_in_air, - mass_fraction_of_hydrophilic_organic_carbon_in_air, - mass_fraction_of_dust001_in_air, mass_fraction_of_dust002_in_air, - mass_fraction_of_dust003_in_air, mass_fraction_of_dust004_in_air, - mass_fraction_of_dust005_in_air, mass_fraction_of_sea_salt001_in_air, - mass_fraction_of_sea_salt002_in_air, mass_fraction_of_sea_salt003_in_air, - mass_fraction_of_sea_salt004_in_air] + state variables: + - mass_fraction_of_sulfate_in_air + - mass_fraction_of_hydrophobic_black_carbon_in_air + - mass_fraction_of_hydrophilic_black_carbon_in_air + - mass_fraction_of_hydrophobic_organic_carbon_in_air + - mass_fraction_of_hydrophilic_organic_carbon_in_air + - mass_fraction_of_dust001_in_air + - mass_fraction_of_dust002_in_air + - mass_fraction_of_dust003_in_air + - mass_fraction_of_dust004_in_air + - mass_fraction_of_dust005_in_air + - mass_fraction_of_sea_salt001_in_air + - mass_fraction_of_sea_salt002_in_air + - mass_fraction_of_sea_salt003_in_air + - mass_fraction_of_sea_salt004_in_air + field io names: + mass_fraction_of_dust001_in_air: dust1 + mass_fraction_of_dust002_in_air: dust2 + mass_fraction_of_dust003_in_air: dust3 + mass_fraction_of_dust004_in_air: dust4 + mass_fraction_of_dust005_in_air: dust5 + mass_fraction_of_sea_salt001_in_air: seas1 + mass_fraction_of_sea_salt002_in_air: seas2 + mass_fraction_of_sea_salt003_in_air: seas3 + mass_fraction_of_sea_salt004_in_air: seas4 + mass_fraction_of_sea_salt005_in_air: seas5 + mass_fraction_of_hydrophobic_black_carbon_in_air: bc1 + mass_fraction_of_hydrophilic_black_carbon_in_air: bc2 + mass_fraction_of_hydrophobic_organic_carbon_in_air: oc1 + mass_fraction_of_hydrophilic_organic_carbon_in_air: oc2 + mass_fraction_of_sulfate_in_air: so4 datapath: {{ aero_background_path }} filename_core: '{{ aero_background_error_time_fv3 }}.anlres.fv_core.res.nc' filename_trcr: '{{ aero_background_error_time_fv3 }}.anlres.fv_tracer.res.nc' diff --git a/parm/jcb-gdas/algorithm/atmosphere/atm_addincrement.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/atm_addincrement.yaml.j2 index 0f0d2ed77..e2fcccec8 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/atm_addincrement.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/atm_addincrement.yaml.j2 @@ -24,13 +24,49 @@ increment geometry: field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_fv3inc.yaml state: datetime: {{ atmosphere_background_time_iso }} - state variables: [ua,va,t,sphum,ice_wat,liq_wat,o3mr,delp,delz] + state variables: + - eastward_wind + - northward_wind + - water_vapor_mixing_ratio_wrt_moist_air + - cloud_liquid_ice + - cloud_liquid_water + - ozone_mass_mixing_ratio + - air_pressure_thickness + - layer_thickness + field io names: + eastward_wind: ugrd + northward_wind: vgrd + air_temperature: tmp + water_vapor_mixing_ratio_wrt_moist_air: spfh + cloud_liquid_ice: icmr + cloud_liquid_water: clwmr + ozone_mass_mixing_ratio: o3mr + air_pressure_thickness: dpres + layer_thickness: delz filetype: cube sphere history provider: ufs filename: {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_atmf006.nc datapath: ./ increment: - added variables: [ua,va,t,sphum,ice_wat,liq_wat,o3mr,delp,delz] + added variables: + - eastward_wind + - northward_wind + - water_vapor_mixing_ratio_wrt_moist_air + - cloud_liquid_ice + - cloud_liquid_water + - ozone_mass_mixing_ratio + - air_pressure_thickness + - layer_thickness + field io names: + eastward_wind: u_inc + northward_wind: v_inc + air_temperature: T_inc + water_vapor_mixing_ratio_wrt_moist_air: sphum_inc + cloud_liquid_ice: icmr_inc + cloud_liquid_water: liq_wat_inc + ozone_mass_mixing_ratio: o3mr_inc + air_pressure_thickness: delp_inc + layer_thickness: delz_inc filetype: fms restart is restart: false filename_nonrestart: {{ atmosphere_variational_analysis_prefix }}cubed_sphere_grid_atminc.nc @@ -39,3 +75,13 @@ output: filetype: auxgrid gridtype: gaussian filename: atmanl. + field io names: + eastward_wind: ugrd + northward_wind: vgrd + air_temperature: tmp + water_vapor_mixing_ratio_wrt_moist_air: spfh + cloud_liquid_ice: icmr + cloud_liquid_water: clwmr + ozone_mass_mixing_ratio: o3mr + air_pressure_thickness: dpres + layer_thickness: delz diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_correction_increment.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_correction_increment.yaml.j2 index 07a9b2119..95eab9c28 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_correction_increment.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_correction_increment.yaml.j2 @@ -1,4 +1,13 @@ -increment variables: [liq_wat,delz,delp,ice_wat,o3mr,sphum,t,ua,va] +increment variables: +- cloud_liquid_water +- layer_thickness +- air_pressure_thickness +- cloud_liquid_ice +- ozone_mass_mixing_ratio +- water_vapor_mixing_ratio_wrt_moist_air +- air_temperature +- eastward_wind +- northward_wind deterministic background geometry: fms initialization: namelist filename: ./fv3jedi/fmsmpp.nml @@ -10,7 +19,6 @@ deterministic background geometry: npx: {{ atmosphere_npx_ges }} npy: {{ atmosphere_npy_ges }} npz: {{ atmosphere_npz_ges }} - field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_history.yaml variational increment geometry: fms initialization: namelist filename: ./fv3jedi/fmsmpp.nml @@ -22,7 +30,6 @@ variational increment geometry: npx: {{ atmosphere_npx_ges }} npy: {{ atmosphere_npy_ges }} npz: {{ atmosphere_npz_ges }} - field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_fv3inc.yaml ensemble mean analysis geometry: fms initialization: namelist filename: ./fv3jedi/fmsmpp.nml @@ -34,7 +41,6 @@ ensemble mean analysis geometry: npx: {{ atmosphere_npx_anl }} npy: {{ atmosphere_npy_anl }} npz: {{ atmosphere_npz_anl }} - field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_history.yaml correction increment geometry: fms initialization: namelist filename: ./fv3jedi/fmsmpp.nml @@ -46,7 +52,6 @@ correction increment geometry: npx: {{ atmosphere_npx_anl }} npy: {{ atmosphere_npy_anl }} npz: {{ atmosphere_npz_anl }} - field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_fv3inc.yaml forecast hours: {% for ihour in range( atmosphere_iau_hours | length ) %} - datetime: {{ atmosphere_iau_times_iso[ihour] }} @@ -55,11 +60,31 @@ forecast hours: provider: ufs datapath: ./ filename: {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_atmf{{ '%03d' % atmosphere_iau_hours[ihour] }}.nc + field io names: + eastward_wind: ugrd + northward_wind: vgrd + air_temperature: tmp + water_vapor_mixing_ratio_wrt_moist_air: spfh + cloud_liquid_ice: icmr + cloud_liquid_water: clwmr + ozone_mass_mixing_ratio: o3mr + air_pressure_thickness: dpres + layer_thickness: delz variational increment: filetype: fms restart is restart: false datapath: ./ filename_nonrestart: {{ atmosphere_variational_analysis_prefix }}cubed_sphere_grid_atmi{{ '%03d' % atmosphere_iau_hours[ihour] }}.nc + field io names: + eastward_wind: u_inc + northward_wind: v_inc + air_temperature: T_inc + water_vapor_mixing_ratio_wrt_moist_air: sphum_inc + cloud_liquid_ice: icmr_inc + cloud_liquid_water: liq_wat_inc + ozone_mass_mixing_ratio: o3mr_inc + air_pressure_thickness: delp_inc + layer_thickness: delz_inc ensemble mean analysis: filetype: cube sphere history provider: ufs @@ -70,4 +95,14 @@ forecast hours: provider: ufs datapath: ./ filename: {{ atmosphere_ensemble_analysis_prefix }}cubed_sphere_grid_catmi{{ '%03d' % atmosphere_iau_hours[ihour] }}.nc + field io names: + eastward_wind: u_inc + northward_wind: v_inc + air_temperature: T_inc + water_vapor_mixing_ratio_wrt_moist_air: sphum_inc + cloud_liquid_ice: icmr_inc + cloud_liquid_water: liq_wat_inc + ozone_mass_mixing_ratio: o3mr_inc + air_pressure_thickness: delp_inc + layer_thickness: delz_inc {% endfor %} diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_ensemble_recenter.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_ensemble_recenter.yaml.j2 index 28382b33a..abbdaf140 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_ensemble_recenter.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_ensemble_recenter.yaml.j2 @@ -12,7 +12,6 @@ members: npx: {{ atmosphere_npx_anl }} npy: {{ atmosphere_npy_anl }} npz: {{ atmosphere_npz_anl }} - field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_fv3inc.yaml increment geometry: fms initialization: namelist filename: ./fv3jedi/fmsmpp.nml @@ -24,16 +23,53 @@ members: npx: {{ atmosphere_npx_anl }} npy: {{ atmosphere_npy_anl }} npz: {{ atmosphere_npz_anl }} - field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_history.yaml state: datetime: {{ atmosphere_iau_times_iso[ihour] }} - state variables: [ua,va,t,sphum,ice_wat,liq_wat,o3mr,delp,delz] + state variables: + - eastward_wind + - northward_wind + - air_temperature + - water_vapor_mixing_ratio_wrt_moist_air + - cloud_liquid_ice + - cloud_liquid_water + - ozone_mass_mixing_ratio + - air_pressure_thickness + - layer_thickness + field io names: + eastward_wind: u_inc + northward_wind: v_inc + air_temperature: T_inc + water_vapor_mixing_ratio_wrt_moist_air: sphum_inc + cloud_liquid_ice: icmr_inc + cloud_liquid_water: liq_wat_inc + ozone_mass_mixing_ratio: o3mr_inc + air_pressure_thickness: delp_inc + layer_thickness: delz_inc filetype: cube sphere history provider: ufs datapath: ./ filename: {{ atmosphere_ensemble_analysis_prefix }}cubed_sphere_grid_catmi{{ '%03d' % atmosphere_iau_hours[ihour] }}.nc increment: - added variables: [ua,va,t,sphum,ice_wat,liq_wat,o3mr,delp,delz] + added variables: + - eastward_wind + - northward_wind + - air_temperature + - water_vapor_mixing_ratio_wrt_moist_air + - cloud_liquid_ice + - cloud_liquid_water + - ozone_mass_mixing_ratio + - air_pressure_thickness + - layer_thickness + field io names: + eastward_wind: u_inc + northward_wind: v_inc + air_temperature: T_inc + water_vapor_mixing_ratio_wrt_moist_air: sphum_inc + cloud_liquid_ice: icmr_inc + cloud_liquid_water: liq_wat_inc + ozone_mass_mixing_ratio: o3mr_inc + air_pressure_thickness: delp_inc + layer_thickness: delz_inc filetype: cube sphere history provider: ufs datapath: ./mem{{ '%0{}d'.format(3)|format(imember+1) }} @@ -43,5 +79,15 @@ members: is restart: false datapath: ./mem{{ '%0{}d'.format(3)|format(imember+1) }} filename_nonrestart: {{ atmosphere_ensemble_analysis_prefix }}cubed_sphere_grid_ratmi{{ '%03d' % atmosphere_iau_hours[ihour] }}.nc + field io names: + eastward_wind: u_inc + northward_wind: v_inc + air_temperature: T_inc + water_vapor_mixing_ratio_wrt_moist_air: sphum_inc + cloud_liquid_ice: icmr_inc + cloud_liquid_water: liq_wat_inc + ozone_mass_mixing_ratio: o3mr_inc + air_pressure_thickness: delp_inc + layer_thickness: delz_inc {% endfor %} {% endfor %} diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 index cac17a0fe..fbb9937ca 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 @@ -1,8 +1,34 @@ variable change: variable change name: Model2GeoVaLs - input variables: &bkgvars [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr,hgtsfc] - output variables: &fv3incrvars [ua,va,t,sphum,ice_wat,liq_wat,o3mr,delp,delz] -jedi increment variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] + input variables: &bkgvars + - eastward_wind + - northward_wind + - air_temperature + - air_pressure_at_surface + - water_vapor_mixing_ratio_wrt_moist_air + - cloud_liquid_ice + - cloud_liquid_water + - ozone_mass_mixing_ratio + - geopotential_height_at_surface + output variables: &fv3incrvars + - eastward_wind + - northward_wind + - air_temperature + - water_vapor_mixing_ratio_wrt_moist_air + - cloud_liquid_ice + - cloud_liquid_water + - ozone_mass_mixing_ratio + - air_pressure_thickness + - layer_thickness +jedi increment variables: +- eastward_wind +- northward_wind +- air_temperature +- air_pressure_at_surface +- water_vapor_mixing_ratio_wrt_moist_air +- cloud_liquid_ice +- cloud_liquid_water +- ozone_mass_mixing_ratio fv3 increment variables: *fv3incrvars background geometry: fms initialization: @@ -15,7 +41,6 @@ background geometry: npx: {{ atmosphere_npx_ges }} npy: {{ atmosphere_npy_ges }} npz: {{ atmosphere_npz_ges }} - field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_history.yaml jedi increment geometry: fms initialization: namelist filename: ./fv3jedi/fmsmpp.nml @@ -27,7 +52,6 @@ jedi increment geometry: npx: {{ atmosphere_npx_ges }} npy: {{ atmosphere_npy_ges }} npz: {{ atmosphere_npz_ges }} - field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_history.yaml fv3 increment geometry: fms initialization: namelist filename: ./fv3jedi/fmsmpp.nml @@ -39,21 +63,33 @@ fv3 increment geometry: npx: {{ atmosphere_npx_ges }} npy: {{ atmosphere_npy_ges }} npz: {{ atmosphere_npz_ges }} - field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_history.yaml members from template: template: background input: datapath: ./bkg/mem%mem% filetype: cube sphere history provider: ufs - ufs soil nlev: 4 - ufs fields split by level: [smc,stc] - filenames: [ {{ atmosphere_ensemble_history_prefix }}cubed_sphere_grid_atmf006.nc, {{ atmosphere_ensemble_history_prefix }}cubed_sphere_grid_sfcf006.nc ] + filenames: + - {{ atmosphere_ensemble_history_prefix }}cubed_sphere_grid_atmf006.nc + - {{ atmosphere_ensemble_history_prefix }}cubed_sphere_grid_sfcf006.nc datetime: '{{ atmosphere_background_time_iso }}' state variables: *bkgvars + field io names: &field_io_names + eastward_wind: ugrd + northward_wind: vgrd + air_temperature: tmp + layer_thickness: delz + air_pressure_thickness: dpres + air_pressure_at_surface: pressfc + water_vapor_mixing_ratio_wrt_moist_air: spfh + cloud_liquid_ice: icmr + cloud_liquid_water: clwmr + ozone_mass_mixing_ratio: o3mr + geopotential_height_at_surface: hgtsfc jedi increment input: filetype: cube sphere history provider: ufs + field io names: *field_io_names datapath: ./anl/mem%mem% filename: cubed_sphere_grid_atminc.jedi.nc fv3 increment output: @@ -61,6 +97,16 @@ members from template: provider: ufs datapath: ./anl/mem%mem% filename: {{ atmosphere_ensemble_analysis_prefix }}cubed_sphere_grid_atminc.nc + field io names: + eastward_wind: u_inc + northward_wind: v_inc + air_temperature: T_inc + water_vapor_mixing_ratio_wrt_moist_air: sphum_inc + cloud_liquid_water: liq_wat_inc + ozone_mass_mixing_ratio: o3mr_inc + cloud_liquid_ice: icmr_inc + air_pressure_thickness: delp_inc + layer_thickness: delz_inc pattern: '%mem%' nmembers: {{ atmosphere_number_ensemble_members }} zero padding: 3 diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 index 91c47f0df..b23d2f65f 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 @@ -1,8 +1,34 @@ variable change: variable change name: Model2GeoVaLs - input variables: &bkgvars [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr,hgtsfc] - output variables: &fv3incrvars [ua,va,t,sphum,ice_wat,liq_wat,o3mr,delp,delz] -jedi increment variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] + input variables: &bkgvars + - eastward_wind + - northward_wind + - air_temperature + - air_pressure_at_surface + - water_vapor_mixing_ratio_wrt_moist_air + - cloud_liquid_ice + - cloud_liquid_water + - ozone_mass_mixing_ratio + - geopotential_height_at_surface + output variables: &fv3incrvars + - eastward_wind + - northward_wind + - air_temperature + - water_vapor_mixing_ratio_wrt_moist_air + - cloud_liquid_ice + - cloud_liquid_water + - ozone_mass_mixing_ratio + - air_pressure_thickness + - layer_thickness +jedi increment variables: +- eastward_wind +- northward_wind +- air_temperature +- air_pressure_at_surface +- water_vapor_mixing_ratio_wrt_moist_air +- cloud_liquid_ice +- cloud_liquid_water +- ozone_mass_mixing_ratio fv3 increment variables: *fv3incrvars background geometry: fms initialization: @@ -15,7 +41,6 @@ background geometry: npx: {{ atmosphere_npx_ges }} npy: {{ atmosphere_npy_ges }} npz: {{ atmosphere_npz_ges }} - field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_history.yaml jedi increment geometry: fms initialization: namelist filename: ./fv3jedi/fmsmpp.nml @@ -27,7 +52,6 @@ jedi increment geometry: npx: {{ atmosphere_npx_ges }} npy: {{ atmosphere_npy_ges }} npz: {{ atmosphere_npz_ges }} - field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_history.yaml fv3 increment geometry: fms initialization: namelist filename: ./fv3jedi/fmsmpp.nml @@ -39,22 +63,32 @@ fv3 increment geometry: npx: {{ atmosphere_npx_ges }} npy: {{ atmosphere_npy_ges }} npz: {{ atmosphere_npz_ges }} - field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_fv3inc.yaml members: - background input: datapath: ./bkg filetype: cube sphere history provider: ufs - ufs soil nlev: 4 - ufs fields split by level: [smc,stc] datetime: "{{ atmosphere_background_time_iso }}" filenames: - {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_atmf006.nc - {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_sfcf006.nc state variables: *bkgvars + field io names: &field_io_names + eastward_wind: ugrd + northward_wind: vgrd + air_temperature: tmp + layer_thickness: delz + air_pressure_thickness: dpres + air_pressure_at_surface: pressfc + water_vapor_mixing_ratio_wrt_moist_air: spfh + cloud_liquid_ice: icmr + cloud_liquid_water: clwmr + ozone_mass_mixing_ratio: o3mr + geopotential_height_at_surface: hgtsfc jedi increment input: filetype: cube sphere history provider: ufs + field io names: *field_io_names datapath: ./anl filename: cubed_sphere_grid_atminc.jedi.nc fv3 increment output: @@ -62,7 +96,16 @@ members: is restart: false datapath: ./anl filename_nonrestart: {{ atmosphere_variational_analysis_prefix }}cubed_sphere_grid_atminc.nc - + field io names: + eastward_wind: u_inc + northward_wind: v_inc + air_temperature: T_inc + water_vapor_mixing_ratio_wrt_moist_air: sphum_inc + cloud_liquid_water: liq_wat_inc + ozone_mass_mixing_ratio: o3mr_inc + cloud_liquid_ice: icmr_inc + air_pressure_thickness: delp_inc + layer_thickness: delz_inc # Optionally test the application {% if do_testing | default(false) %} test: diff --git a/parm/jcb-gdas/algorithm/snow/fv3jedi_land_ensrecenter.yaml.j2 b/parm/jcb-gdas/algorithm/snow/fv3jedi_land_ensrecenter.yaml.j2 index 81d6f9ce7..5d1eb1630 100644 --- a/parm/jcb-gdas/algorithm/snow/fv3jedi_land_ensrecenter.yaml.j2 +++ b/parm/jcb-gdas/algorithm/snow/fv3jedi_land_ensrecenter.yaml.j2 @@ -9,17 +9,19 @@ geometry: npx: {{ npx_ges }} npy: {{ npy_ges }} npz: {{ npz_ges }} - field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_restart.yaml date: "{{ snow_background_time_iso }}" -variables: [totalSnowDepth] +variables: +- totalSnowDepth increment mask: - variable: [slmsk] + variable: + - slmsk minvalue: 0.99 maxvalue: 1.01 background: datetime: "{{ snow_background_time_iso }}" filetype: fms restart - state variables: [slmsk] + state variables: + - slmsk datapath: ./bkg/mem001 filename is datetime templated: true filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" @@ -31,7 +33,10 @@ increment mask: deterministic increment: datetime: "{{ snow_increment_time_iso }}" filetype: fms restart - state variables: [snodl] + state variables: + - totalSnowDepth + field io names: + totalSnowDepth: snodl datapath: ./inc/det_ensres filename is datetime templated: true filename_core: "snowinc.%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" @@ -43,7 +48,10 @@ deterministic increment: deterministic background: datetime: "{{ snow_background_time_iso }}" filetype: fms restart - state variables: [snodl] + state variables: + - totalSnowDepth + field io names: + totalSnowDepth: snodl datapath: ./bkg/det_ensres filename is datetime templated: true filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" @@ -57,6 +65,8 @@ output increment: datapath: ./inc/ensmean datetime: "{{ snow_increment_time_iso }}" filetype: fms restart + field io names: + totalSnowDepth: snodl prefix: snowinc filename_core: "{{ snow_increment_time_fv3 }}.fv_core.res.nc" filename_trcr: "{{ snow_increment_time_fv3 }}.fv_tracer.res.nc" @@ -70,7 +80,10 @@ ensemble backgrounds: template: datetime: "{{ snow_background_time_iso }}" filetype: fms restart - state variables: [snodl] + state variables: + - totalSnowDepth + field io names: + totalSnowDepth: snodl datapath: ./bkg/mem%mem% filename is datetime templated: true filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" diff --git a/parm/jcb-gdas/algorithm/snow/fv3jedi_snow_ensmean.yaml.j2 b/parm/jcb-gdas/algorithm/snow/fv3jedi_snow_ensmean.yaml.j2 index b48929e12..9bd29e33a 100644 --- a/parm/jcb-gdas/algorithm/snow/fv3jedi_snow_ensmean.yaml.j2 +++ b/parm/jcb-gdas/algorithm/snow/fv3jedi_snow_ensmean.yaml.j2 @@ -11,12 +11,35 @@ mean output: filename_sfcd: sfc_data.nc filename_sfcw: fv_srf_wnd.res.nc filename_cplr: coupler.res + state variables: + - totalSnowDepth + - vtype + - slmsk + - filtered_orography + - stc + - sheleg + - fraction_of_ice + field io names: + totalSnowDepth: snodl + filtered_orography: orog_filt + fraction_of_ice: fice ensemble: members from template: template: datetime: '{{ snow_background_time_iso }}' filetype: fms restart - state variables: [snodl,vtype,slmsk,orog_filt,stc,sheleg,fice] + state variables: + - totalSnowDepth + - vtype + - slmsk + - filtered_orography + - stc + - sheleg + - fraction_of_ice + field io names: + totalSnowDepth: snodl + filtered_orography: orog_filt + fraction_of_ice: fice datapath: ./bkg/mem%mem%/ filename_sfcd: '{{ snow_background_time_fv3 }}.sfc_data.nc' filename_cplr: '{{ snow_background_time_fv3 }}.coupler.res' diff --git a/parm/jcb-gdas/model/aero/aero_3dvar_outer_loop_1.yaml.j2 b/parm/jcb-gdas/model/aero/aero_3dvar_outer_loop_1.yaml.j2 index 68529a1be..2c6183651 100644 --- a/parm/jcb-gdas/model/aero/aero_3dvar_outer_loop_1.yaml.j2 +++ b/parm/jcb-gdas/model/aero/aero_3dvar_outer_loop_1.yaml.j2 @@ -12,6 +12,5 @@ npx: {{aero_npx_anl}} npy: {{aero_npy_anl}} npz: {{aero_npz_anl}} - field metadata override: {{aero_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml diagnostics: departures: bkgmob diff --git a/parm/jcb-gdas/model/aero/aero_3dvar_outer_loop_2.yaml.j2 b/parm/jcb-gdas/model/aero/aero_3dvar_outer_loop_2.yaml.j2 index 7ca1a03f1..edd109797 100644 --- a/parm/jcb-gdas/model/aero/aero_3dvar_outer_loop_2.yaml.j2 +++ b/parm/jcb-gdas/model/aero/aero_3dvar_outer_loop_2.yaml.j2 @@ -12,6 +12,5 @@ npx: {{aero_npx_anl}} npy: {{aero_npy_anl}} npz: {{aero_npz_anl}} - field metadata override: {{aero_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml diagnostics: departures: bkgmob1 diff --git a/parm/jcb-gdas/model/aero/aero_background.yaml.j2 b/parm/jcb-gdas/model/aero/aero_background.yaml.j2 index 012c50db5..85e796a24 100644 --- a/parm/jcb-gdas/model/aero/aero_background.yaml.j2 +++ b/parm/jcb-gdas/model/aero/aero_background.yaml.j2 @@ -5,6 +5,41 @@ filename is datetime templated: true filename_core: '%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc' filename_trcr: '%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc' filename_cplr: '%yyyy%mm%dd.%hh%MM%ss.coupler.res' -state variables: [t,delp,sphum,so4,bc1,bc2,oc1,oc2, - dust1,dust2,dust3,dust4,dust5, - seas1,seas2,seas3,seas4] +state variables: +- air_temperature +- air_pressure_thickness +- water_vapor_mixing_ratio_wrt_moist_air +- mass_fraction_of_sulfate_in_air +- mass_fraction_of_hydrophobic_black_carbon_in_air +- mass_fraction_of_hydrophilic_black_carbon_in_air +- mass_fraction_of_hydrophobic_organic_carbon_in_air +- mass_fraction_of_hydrophilic_organic_carbon_in_air +- mass_fraction_of_dust001_in_air +- mass_fraction_of_dust002_in_air +- mass_fraction_of_dust003_in_air +- mass_fraction_of_dust004_in_air +- mass_fraction_of_dust005_in_air +- mass_fraction_of_sea_salt001_in_air +- mass_fraction_of_sea_salt002_in_air +- mass_fraction_of_sea_salt003_in_air +- mass_fraction_of_sea_salt004_in_air +- mass_fraction_of_sea_salt005_in_air +field io names: + air_temperature: T + air_pressure_thickness: delp + water_vapor_mixing_ratio_wrt_moist_air: sphum + mass_fraction_of_sulfate_in_air: so4 + mass_fraction_of_hydrophobic_black_carbon_in_air: bc1 + mass_fraction_of_hydrophilic_black_carbon_in_air: bc2 + mass_fraction_of_hydrophobic_organic_carbon_in_air: oc1 + mass_fraction_of_hydrophilic_organic_carbon_in_air: oc2 + mass_fraction_of_dust001_in_air: dust1 + mass_fraction_of_dust002_in_air: dust2 + mass_fraction_of_dust003_in_air: dust3 + mass_fraction_of_dust004_in_air: dust4 + mass_fraction_of_dust005_in_air: dust5 + mass_fraction_of_sea_salt001_in_air: seas1 + mass_fraction_of_sea_salt002_in_air: seas2 + mass_fraction_of_sea_salt003_in_air: seas3 + mass_fraction_of_sea_salt004_in_air: seas4 + mass_fraction_of_sea_salt005_in_air: seas5 diff --git a/parm/jcb-gdas/model/aero/aero_background_error_static_diffusion.yaml.j2 b/parm/jcb-gdas/model/aero/aero_background_error_static_diffusion.yaml.j2 index 73af01bb8..02de0dfa4 100644 --- a/parm/jcb-gdas/model/aero/aero_background_error_static_diffusion.yaml.j2 +++ b/parm/jcb-gdas/model/aero/aero_background_error_static_diffusion.yaml.j2 @@ -1,23 +1,28 @@ covariance model: SABER saber central block: - saber block name: diffusion + saber block name: diffusion read: groups: - - variables: [mass_fraction_of_sulfate_in_air, - mass_fraction_of_hydrophobic_black_carbon_in_air, - mass_fraction_of_hydrophilic_black_carbon_in_air, - mass_fraction_of_hydrophobic_organic_carbon_in_air, - mass_fraction_of_hydrophilic_organic_carbon_in_air, - mass_fraction_of_dust001_in_air, mass_fraction_of_dust002_in_air, - mass_fraction_of_dust003_in_air, mass_fraction_of_dust004_in_air, - mass_fraction_of_dust005_in_air, mass_fraction_of_sea_salt001_in_air, - mass_fraction_of_sea_salt002_in_air, mass_fraction_of_sea_salt003_in_air, - mass_fraction_of_sea_salt004_in_air] + - variables: + - mass_fraction_of_sulfate_in_air + - mass_fraction_of_hydrophobic_black_carbon_in_air + - mass_fraction_of_hydrophilic_black_carbon_in_air + - mass_fraction_of_hydrophobic_organic_carbon_in_air + - mass_fraction_of_hydrophilic_organic_carbon_in_air + - mass_fraction_of_dust001_in_air + - mass_fraction_of_dust002_in_air + - mass_fraction_of_dust003_in_air + - mass_fraction_of_dust004_in_air + - mass_fraction_of_dust005_in_air + - mass_fraction_of_sea_salt001_in_air + - mass_fraction_of_sea_salt002_in_air + - mass_fraction_of_sea_salt003_in_air + - mass_fraction_of_sea_salt004_in_air horizontal: filepath: "{{aero_berror_data_directory}}/diffusion_hz" vertical: - levels: {{aero_npz_anl}} - filepath: "{{aero_berror_data_directory}}/diffusion_vt" + levels: {{aero_npz_anl}} + filepath: "{{aero_berror_data_directory}}/diffusion_vt" saber outer blocks: - saber block name: StdDev read: @@ -30,4 +35,20 @@ saber outer blocks: filename_core: '{{ aero_cycle_time_fv3 }}.stddev.fv_core.res.nc' filename_trcr: '{{ aero_cycle_time_fv3 }}.stddev.fv_tracer.res.nc' filename_cplr: '{{ aero_cycle_time_fv3 }}.stddev.coupler.res' - date: "{{ aero_cycle_time_iso }}" + date: "{{ aero_cycle_time_iso }}" + field io names: + mass_fraction_of_dust001_in_air: dust1 + mass_fraction_of_dust002_in_air: dust2 + mass_fraction_of_dust003_in_air: dust3 + mass_fraction_of_dust004_in_air: dust4 + mass_fraction_of_dust005_in_air: dust5 + mass_fraction_of_sea_salt001_in_air: seas1 + mass_fraction_of_sea_salt002_in_air: seas2 + mass_fraction_of_sea_salt003_in_air: seas3 + mass_fraction_of_sea_salt004_in_air: seas4 + mass_fraction_of_sea_salt005_in_air: seas5 + mass_fraction_of_hydrophobic_black_carbon_in_air: bc1 + mass_fraction_of_hydrophilic_black_carbon_in_air: bc2 + mass_fraction_of_hydrophobic_organic_carbon_in_air: oc1 + mass_fraction_of_hydrophilic_organic_carbon_in_air: oc2 + mass_fraction_of_sulfate_in_air: so4 diff --git a/parm/jcb-gdas/model/aero/aero_final_increment_cubed_sphere.yaml.j2 b/parm/jcb-gdas/model/aero/aero_final_increment_cubed_sphere.yaml.j2 index d489b2b70..9b3ebe318 100644 --- a/parm/jcb-gdas/model/aero/aero_final_increment_cubed_sphere.yaml.j2 +++ b/parm/jcb-gdas/model/aero/aero_final_increment_cubed_sphere.yaml.j2 @@ -9,7 +9,6 @@ geometry: npx: {{aero_npx_ges}} npy: {{aero_npy_ges}} npz: {{aero_npz_ges}} - field metadata override: "{{aero_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" output: state component: datapath: {{ aero_analysis_path }} @@ -18,6 +17,39 @@ output: filename_core: '{{ aero_cycle_time_fv3 }}.fv_core.res.nc' filename_trcr: '{{ aero_cycle_time_fv3 }}.fv_tracer.res.nc' filename_cplr: '{{ aero_cycle_time_fv3 }}.coupler.res' - state variables: [t,delp,sphum,so4,bc1,bc2,oc1,oc2, - dust1,dust2,dust3,dust4,dust5, - seas1,seas2,seas3,seas4] + state variables: + - air_temperature + - air_pressure_thickness + - water_vapor_mixing_ratio_wrt_moist_air + - mass_fraction_of_sulfate_in_air + - mass_fraction_of_hydrophobic_black_carbon_in_air + - mass_fraction_of_hydrophilic_black_carbon_in_air + - mass_fraction_of_hydrophobic_organic_carbon_in_air + - mass_fraction_of_hydrophilic_organic_carbon_in_air + - mass_fraction_of_dust001_in_air + - mass_fraction_of_dust002_in_air + - mass_fraction_of_dust003_in_air + - mass_fraction_of_dust004_in_air + - mass_fraction_of_dust005_in_air + - mass_fraction_of_sea_salt001_in_air + - mass_fraction_of_sea_salt002_in_air + - mass_fraction_of_sea_salt003_in_air + - mass_fraction_of_sea_salt004_in_air + field io names: + air_temperature: T + air_pressure_thickness: delp + water_vapor_mixing_ratio_wrt_moist_air: sphum + mass_fraction_of_dust001_in_air: dust1 + mass_fraction_of_dust002_in_air: dust2 + mass_fraction_of_dust003_in_air: dust3 + mass_fraction_of_dust004_in_air: dust4 + mass_fraction_of_dust005_in_air: dust5 + mass_fraction_of_sea_salt001_in_air: seas1 + mass_fraction_of_sea_salt002_in_air: seas2 + mass_fraction_of_sea_salt003_in_air: seas3 + mass_fraction_of_sea_salt004_in_air: seas4 + mass_fraction_of_hydrophobic_black_carbon_in_air: bc1 + mass_fraction_of_hydrophilic_black_carbon_in_air: bc2 + mass_fraction_of_hydrophobic_organic_carbon_in_air: oc1 + mass_fraction_of_hydrophilic_organic_carbon_in_air: oc2 + mass_fraction_of_sulfate_in_air: so4 diff --git a/parm/jcb-gdas/model/aero/aero_final_increment_gaussian.yaml.j2 b/parm/jcb-gdas/model/aero/aero_final_increment_gaussian.yaml.j2 index 72eabe067..87f0b689f 100644 --- a/parm/jcb-gdas/model/aero/aero_final_increment_gaussian.yaml.j2 +++ b/parm/jcb-gdas/model/aero/aero_final_increment_gaussian.yaml.j2 @@ -3,16 +3,21 @@ local interpolator type: atlas interpolator interpolation method: type: finite-element number of latitude gridpoints: {{ aero_npy_anl - 1 }} -variables to output: [mass_fraction_of_sulfate_in_air, - mass_fraction_of_hydrophobic_black_carbon_in_air, - mass_fraction_of_hydrophilic_black_carbon_in_air, - mass_fraction_of_hydrophobic_organic_carbon_in_air, - mass_fraction_of_hydrophilic_organic_carbon_in_air, - mass_fraction_of_dust001_in_air, mass_fraction_of_dust002_in_air, - mass_fraction_of_dust003_in_air, mass_fraction_of_dust004_in_air, - mass_fraction_of_dust005_in_air, mass_fraction_of_sea_salt001_in_air, - mass_fraction_of_sea_salt002_in_air, mass_fraction_of_sea_salt003_in_air, - mass_fraction_of_sea_salt004_in_air] +variables to output: +- mass_fraction_of_sulfate_in_air +- mass_fraction_of_hydrophobic_black_carbon_in_air +- mass_fraction_of_hydrophilic_black_carbon_in_air +- mass_fraction_of_hydrophobic_organic_carbon_in_air +- mass_fraction_of_hydrophilic_organic_carbon_in_air +- mass_fraction_of_dust001_in_air +- mass_fraction_of_dust002_in_air +- mass_fraction_of_dust003_in_air +- mass_fraction_of_dust004_in_air +- mass_fraction_of_dust005_in_air +- mass_fraction_of_sea_salt001_in_air +- mass_fraction_of_sea_salt002_in_air +- mass_fraction_of_sea_salt003_in_air +- mass_fraction_of_sea_salt004_in_air all model levels: true datapath: {{ aero_analysis_path }} prefix: aeroinc_gauss diff --git a/parm/jcb-gdas/model/aero/aero_geometry_background.yaml.j2 b/parm/jcb-gdas/model/aero/aero_geometry_background.yaml.j2 index 834e2fcca..b55f395ab 100644 --- a/parm/jcb-gdas/model/aero/aero_geometry_background.yaml.j2 +++ b/parm/jcb-gdas/model/aero/aero_geometry_background.yaml.j2 @@ -8,4 +8,3 @@ layout: npx: {{aero_npx_ges}} npy: {{aero_npy_ges}} npz: {{aero_npz_ges}} -field metadata override: "{{aero_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" diff --git a/parm/jcb-gdas/model/aero/aero_model_pseudo.yaml.j2 b/parm/jcb-gdas/model/aero/aero_model_pseudo.yaml.j2 index 61af8db2a..149db9181 100644 --- a/parm/jcb-gdas/model/aero/aero_model_pseudo.yaml.j2 +++ b/parm/jcb-gdas/model/aero/aero_model_pseudo.yaml.j2 @@ -7,7 +7,42 @@ filename_trcr: '%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc' filename_sfcd: '%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc' filename_sfcw: '%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc' filename_cplr: '%yyyy%mm%dd.%hh%MM%ss.coupler.res' +state variables: +- air_temperature +- air_pressure_thickness +- water_vapor_mixing_ratio_wrt_moist_air +- mass_fraction_of_sulfate_in_air +- mass_fraction_of_hydrophobic_black_carbon_in_air +- mass_fraction_of_hydrophilic_black_carbon_in_air +- mass_fraction_of_hydrophobic_organic_carbon_in_air +- mass_fraction_of_hydrophilic_organic_carbon_in_air +- mass_fraction_of_dust001_in_air +- mass_fraction_of_dust002_in_air +- mass_fraction_of_dust003_in_air +- mass_fraction_of_dust004_in_air +- mass_fraction_of_dust005_in_air +- mass_fraction_of_sea_salt001_in_air +- mass_fraction_of_sea_salt002_in_air +- mass_fraction_of_sea_salt003_in_air +- mass_fraction_of_sea_salt004_in_air +- mass_fraction_of_sea_salt005_in_air +field io names: + air_temperature: T + air_pressure_thickness: delp + water_vapor_mixing_ratio_wrt_moist_air: sphum + mass_fraction_of_sulfate_in_air: so4 + mass_fraction_of_hydrophobic_black_carbon_in_air: bc1 + mass_fraction_of_hydrophilic_black_carbon_in_air: bc2 + mass_fraction_of_hydrophobic_organic_carbon_in_air: oc1 + mass_fraction_of_hydrophilic_organic_carbon_in_air: oc2 + mass_fraction_of_dust001_in_air: dust1 + mass_fraction_of_dust002_in_air: dust2 + mass_fraction_of_dust003_in_air: dust3 + mass_fraction_of_dust004_in_air: dust4 + mass_fraction_of_dust005_in_air: dust5 + mass_fraction_of_sea_salt001_in_air: seas1 + mass_fraction_of_sea_salt002_in_air: seas2 + mass_fraction_of_sea_salt003_in_air: seas3 + mass_fraction_of_sea_salt004_in_air: seas4 + mass_fraction_of_sea_salt005_in_air: seas5 tstep: {{aero_forecast_timestep}} -model variables: [t,delp,sphum,so4,bc1,bc2,oc1,oc2, - dust1,dust2,dust3,dust4,dust5, - seas1,seas2,seas3,seas4] diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 index 671fbd3e9..3b2c58cb5 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 @@ -12,7 +12,6 @@ npx: {{atmosphere_npx_anl}} npy: {{atmosphere_npy_anl}} npz: {{atmosphere_npz_anl}} - field metadata override: {{atmosphere_fv3jedi_files_path}}/fv3jedi_fieldmetadata_history.yaml vert coordinate: logp diagnostics: departures: bkgmob diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 index 86344aecb..93de3f1ba 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 @@ -12,7 +12,6 @@ npx: {{atmosphere_npx_anl}} npy: {{atmosphere_npy_anl}} npz: {{atmosphere_npz_anl}} - field metadata override: {{atmosphere_fv3jedi_files_path}}/fv3jedi_fieldmetadata_history.yaml vert coordinate: logp diagnostics: departures: bkgmob1 diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 index 6c2d847f4..86ea4014e 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 @@ -2,7 +2,61 @@ datapath: {{ atmosphere_background_path }} filetype: cube sphere history provider: ufs datetime: "{{ atmosphere_background_time_iso }}" -filenames: [ {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_atmf006.nc, {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_sfcf006.nc ] -state variables: [ua,va,t,delp,ps,sphum,ice_wat,liq_wat,snmr,rwmr,grle,nccice,nconrd,o3mr,hgtsfc, - slmsk,sheleg,tmpsfc,vtype,stype,vfrac,soilt1,soilw1,snwdphMeters, - u_srf,v_srf,f10m] +filenames: +- {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_atmf006.nc +- {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_sfcf006.nc +state variables: +- eastward_wind +- northward_wind +- air_temperature +- air_pressure_thickness +- air_pressure_at_surface +- water_vapor_mixing_ratio_wrt_moist_air +- cloud_liquid_ice +- cloud_liquid_water +- snow_water +- rain_water +- graupel +- cloud_ice_number_concentration +- rain_number_concentration +- ozone_mass_mixing_ratio +- geopotential_height_at_surface +- slmsk +- sheleg +- skin_temperature_at_surface +- vtype +- stype +- vfrac +- soilt +- soilm +- totalSnowDepthMeters +- eastward_wind_at_surface +- northward_wind_at_surface +- f10m +field io names: + eastward_wind: ugrd + northward_wind: vgrd + air_temperature: tmp + layer_thickness: delz + air_pressure_thickness: dpres + air_pressure_at_surface: pressfc + water_vapor_mixing_ratio_wrt_moist_air: spfh + cloud_liquid_ice: icmr + cloud_liquid_water: clwmr + snow_water: snmr + rain_water: rwmr + graupel: grle + cloud_ice_number_concentration: nccice + rain_number_concentration: nconrd + ozone_mass_mixing_ratio: o3mr + geopotential_height_at_surface: hgtsfc + slmsk: land + sheleg: weasd + skin_temperature_at_surface: tmpsfc + stype: sotyp + vfrac: veg + totalSnowDepthMeters: snod + eastward_wind_at_surface: ugrd_hyblev1 + northward_wind_at_surface: vgrd_hyblev1 + soilm: soilw1 + soilt: soilt1 diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_background_ensemble.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background_ensemble.yaml.j2 index 6e7902a72..7ee0f97c0 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_background_ensemble.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background_ensemble.yaml.j2 @@ -4,12 +4,67 @@ members from template: datetime: '{{ atmosphere_background_time_iso }}' filetype: cube sphere history provider: ufs - state variables: [ua,va,t,delz,delp,ps,sphum,ice_wat,liq_wat,snmr,rwmr,grle,nccice,nconrd,o3mr,hgtsfc, - slmsk,sheleg,tmpsfc,vtype,stype,vfrac,soilt1,soilw1,snwdphMeters, - u_srf,v_srf,f10m] + state variables: + - eastward_wind + - northward_wind + - air_temperature + - layer_thickness + - air_pressure_thickness + - air_pressure_at_surface + - water_vapor_mixing_ratio_wrt_moist_air + - cloud_liquid_ice + - cloud_liquid_water + - snow_water + - rain_water + - graupel + - cloud_ice_number_concentration + - rain_number_concentration + - ozone_mass_mixing_ratio + - geopotential_height_at_surface + - slmsk + - sheleg + - skin_temperature_at_surface + - vtype + - stype + - vfrac + - soilt + - soilm + - totalSnowDepthMeters + - eastward_wind_at_surface + - northward_wind_at_surface + - f10m + field io names: + eastward_wind: ugrd + northward_wind: vgrd + air_temperature: tmp + layer_thickness: delz + air_pressure_thickness: dpres + air_pressure_at_surface: pressfc + water_vapor_mixing_ratio_wrt_moist_air: spfh + cloud_liquid_ice: icmr + cloud_liquid_water: clwmr + snow_water: snmr + rain_water: rwmr + graupel: grle + cloud_ice_number_concentration: nccice + rain_number_concentration: nconrd + ozone_mass_mixing_ratio: o3mr + geopotential_height_at_surface: hgtsfc + slmsk: land + sheleg: weasd + skin_temperature_at_surface: tmpsfc + stype: sotyp + vfrac: veg + totalSnowDepthMeters: snod + eastward_wind_at_surface: ugrd_hyblev1 + northward_wind_at_surface: vgrd_hyblev1 + soilm: soilw1 + soilt: soilt1 datapath: {{ atmosphere_background_ensemble_path }} filename is datetime templated: true - filenames: [ {{ atmosphere_ensemble_history_prefix }}cubed_sphere_grid_atmf006.nc, {{ atmosphere_ensemble_history_prefix }}cubed_sphere_grid_sfcf006.nc ] + filenames: + - {{ atmosphere_ensemble_history_prefix }}cubed_sphere_grid_atmf006.nc + - {{ atmosphere_ensemble_history_prefix }}cubed_sphere_grid_sfcf006.nc pattern: "{{ atmosphere_ensemble_pattern | default("%mem%", true) }}" nmembers: {{ atmosphere_number_ensemble_members }} zero padding: 3 diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_bump_bump.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_bump_bump.yaml.j2 deleted file mode 100644 index bf30ace2b..000000000 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_bump_bump.yaml.j2 +++ /dev/null @@ -1,128 +0,0 @@ -covariance model: hybrid -components: -- covariance: - covariance model: SABER - saber blocks: - - saber block name: BUMP_NICAS - saber central block: true - input variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] - output variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] - active variables: [psi,chi,t,ps,rh,liq_wat,o3mr] - bump: - io: - data directory: {{atmosphere_bump_data_directory}} - drivers: - multivariate strategy: specific_univariate - read local nicas: true - nicas: - minimum level: - - variables: - - liq_wat - value: 76 - grids: - - model: - variables: - - stream_function - - velocity_potential - - air_temperature - - relative_humidity - - cloud_liquid_water - - ozone_mass_mixing_ratio - io: - files prefix: nicas/nicas_3D_gfs - - model: - variables: - - surface_pressure - io: - files prefix: nicas/nicas_2D_gfs - - saber block name: StdDev - input variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] - output variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] - active variables: [psi,chi,t,ps,rh,liq_wat,o3mr] - file: - filetype: fms restart - datetime: 2021-12-22T00:00:00Z - set datetime on read: true - psinfile: true - datapath: {{atmosphere_bump_data_directory}} - prefix: stddev/20211222.000000 - filename_core: stddev.fv_core.res.nc - filename_trcr: stddev.fv_tracer.res.nc - filename_sfcd: stddev.sfc_data.nc - filename_sfcw: stddev.fv_srf_wnd.res.nc - filename_cplr: stddev.coupler.res - - saber block name: BUMP_VerticalBalance - input variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] - output variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] - active variables: [psi,chi,t,ps,rh,liq_wat,o3mr] - bump: - general: - universe length-scale: 2000.0e3 - io: - data directory: {{atmosphere_bump_data_directory}} - files prefix: vbal/vbal_gfs - overriding sampling file: vbal/vbal_2021122200_gfs_sampling - drivers: - read local sampling: true - read vertical balance: true - vertical balance: - vbal: - - balanced variable: var2 - unbalanced variable: var1 - - balanced variable: var3 - unbalanced variable: var1 - - balanced variable: var4 - unbalanced variable: var1 - - saber block name: BUMP_PsiChiToUV - input variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] - output variables: [ua,va,t,ps,rh,ice_wat,liq_wat,o3mr] - active variables: [psi,chi,ua,va] - bump: - general: - universe length-scale: 2000.0e3 - io: - data directory: {{atmosphere_bump_data_directory}} - files prefix: psichitouv/psichitouv_gfs - drivers: - read local psichitouv: true - linear variable change: - linear variable change name: Control2Analysis - input variables: [ua,va,t,ps,rh,ice_wat,liq_wat,o3mr] - output variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] - weight: - value: 0.1 -- covariance: - covariance model: ensemble - members from template: - template: - datetime: "{{ atmosphere_background_time_iso }}" - filetype: fms restart - state variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] - datapath: {{ atmosphere_background_ensemble_path}} - filename is datetime templated: true - filename_core: "%yyyy%mm%dd.%hh%MM%ss.fv_core.res.nc" - filename_trcr: "%yyyy%mm%dd.%hh%MM%ss.fv_tracer.res.nc" - filename_sfcd: "%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc" - filename_sfcw: "%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc" - filename_cplr: "%yyyy%mm%dd.%hh%MM%ss.coupler.res" - pattern: "{{ atmosphere_ensemble_pattern | default("%mem%", true) }}" - nmembers: {{atmosphere_number_ensemble_members}} - zero padding: 3 -# localization: -# localization method: SABER -# saber block: -# - saber block name: BUMP_NICAS -# input variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] -# output variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] -# active variables: [psi,chi,t,ps,rh,liq_wat,o3mr] -# bump: -# datadir: {{atmosphere_bump_data_directory}} -# verbosity: main -# strategy: specific_univariate -# method: loc -# load_nicas_local: true -# grids: -# - prefix: nicas/nicas_3D_gfs -# variables: [stream_function,velocity_potential,air_temperature,relative_humidity,cloud_liquid_water,ozone_mass_mixing_ratio] - weight: - value: 0.9 diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 index 88ce07287..9f6a59f36 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 @@ -32,10 +32,24 @@ components: ozone_mass_mixing_ratio] linear variable change: linear variable change name: Control2Analysis - input variables: [eastward_wind,northward_wind,air_temperature,air_pressure_at_surface, - water_vapor_mixing_ratio_wrt_moist_air,cloud_liquid_ice,cloud_liquid_water, - mole_fraction_of_ozone_in_air] - output variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] + input variables: + - eastward_wind + - northward_wind + - air_temperature + - air_pressure_at_surface + - water_vapor_mixing_ratio_wrt_moist_air + - cloud_liquid_ice + - cloud_liquid_water + - mole_fraction_of_ozone_in_air + output variables: + - eastward_wind + - northward_wind + - air_temperature + - air_pressure_at_surface + - water_vapor_mixing_ratio_wrt_moist_air + - cloud_liquid_ice + - cloud_liquid_water + - ozone_mass_mixing_ratio weight: value: 0.125 - covariance: @@ -45,10 +59,29 @@ components: datetime: "{{ atmosphere_background_time_iso }}" filetype: cube sphere history provider: ufs - state variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] + state variables: + - eastward_wind + - northward_wind + - air_temperature + - air_pressure_at_surface + - water_vapor_mixing_ratio_wrt_moist_air + - cloud_liquid_ice + - cloud_liquid_water + - ozone_mass_mixing_ratio + field io names: + eastward_wind: ugrd + northward_wind: vgrd + air_temperature: tmp + air_pressure_at_surface: pressfc + water_vapor_mixing_ratio_wrt_moist_air: spfh + cloud_liquid_ice: icmr + cloud_liquid_water: clwmr + ozone_mass_mixing_ratio: o3mr datapath: {{ atmosphere_background_ensemble_path }} filename is datetime templated: true - filenames: [ {{ atmosphere_ensemble_history_prefix }}cubed_sphere_grid_atmf006.nc, {{ atmosphere_ensemble_history_prefix }}cubed_sphere_grid_sfcf006.nc ] + filenames: + - {{ atmosphere_ensemble_history_prefix }}cubed_sphere_grid_atmf006.nc + - {{ atmosphere_ensemble_history_prefix }}cubed_sphere_grid_sfcf006.nc pattern: "{{ atmosphere_ensemble_pattern | default("%mem%", true) }}" nmembers: {{atmosphere_number_ensemble_members}} zero padding: 3 @@ -56,9 +89,15 @@ components: localization method: SABER saber central block: saber block name: BUMP_NICAS - active variables: [eastward_wind,northward_wind,air_temperature,air_pressure_at_surface, - water_vapor_mixing_ratio_wrt_moist_air,cloud_liquid_ice,cloud_liquid_water, - ozone_mass_mixing_ratio ] + active variables: + - eastward_wind + - northward_wind + - air_temperature + - air_pressure_at_surface + - water_vapor_mixing_ratio_wrt_moist_air + - cloud_liquid_ice + - cloud_liquid_water + - ozone_mass_mixing_ratio read: general: universe length-scale: 4641.0e3 @@ -207,9 +246,23 @@ components: value: 1.8 linear variable change: linear variable change name: Control2Analysis - input variables: [eastward_wind,northward_wind,air_temperature,air_pressure_at_surface, - water_vapor_mixing_ratio_wrt_moist_air,cloud_liquid_ice,cloud_liquid_water, - ozone_mass_mixing_ratio ] - output variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] + input variables: + - eastward_wind + - northward_wind + - air_temperature + - air_pressure_at_surface + - water_vapor_mixing_ratio_wrt_moist_air + - cloud_liquid_ice + - cloud_liquid_water + - ozone_mass_mixing_ratio + output variables: + - eastward_wind + - northward_wind + - air_temperature + - air_pressure_at_surface + - water_vapor_mixing_ratio_wrt_moist_air + - cloud_liquid_ice + - cloud_liquid_water + - ozone_mass_mixing_ratio weight: value: 0.875 diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_bump.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_bump.yaml.j2 deleted file mode 100644 index 68601e969..000000000 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_bump.yaml.j2 +++ /dev/null @@ -1,89 +0,0 @@ -covariance model: SABER -full inverse: true -saber blocks: -- saber block name: BUMP_NICAS - saber central block: true - input variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] - output variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] - active variables: [psi,chi,t,ps,rh,liq_wat,o3mr] - bump: - io: - data directory: {{atmosphere_bump_data_directory}} - drivers: - multivariate strategy: specific_univariate - read local nicas: true - nicas: - minimum level: - - variables: - - liq_wat - value: 76 - grids: - - model: - variables: - - stream_function - - velocity_potential - - air_temperature - - relative_humidity - - cloud_liquid_water - - ozone_mass_mixing_ratio - io: - files prefix: nicas/nicas_3D_gfs - - model: - variables: - - surface_pressure - io: - files prefix: nicas/nicas_2D_gfs -- saber block name: StdDev - input variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] - output variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] - active variables: [psi,chi,t,ps,rh,liq_wat,o3mr] - file: - filetype: fms restart - datetime: 2021-12-22T00:00:00Z - set datetime on read: true - psinfile: true - datapath: {{atmosphere_bump_data_directory}} - prefix: stddev/20211222.000000 - filename_core: stddev.fv_core.res.nc - filename_trcr: stddev.fv_tracer.res.nc - filename_sfcd: stddev.sfc_data.nc - filename_sfcw: stddev.fv_srf_wnd.res.nc - filename_cplr: stddev.coupler.res -- saber block name: BUMP_VerticalBalance - input variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] - output variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] - active variables: [psi,chi,t,ps,rh,liq_wat,o3mr] - bump: - general: - universe length-scale: 2000.0e3 - io: - data directory: {{atmosphere_bump_data_directory}} - files prefix: vbal/vbal_gfs - overriding sampling file: vbal/vbal_2021122200_gfs_sampling - drivers: - read local sampling: true - read vertical balance: true - vertical balance: - vbal: - - balanced variable: var2 - unbalanced variable: var1 - - balanced variable: var3 - unbalanced variable: var1 - - balanced variable: var4 - unbalanced variable: var1 -- saber block name: BUMP_PsiChiToUV - input variables: [psi,chi,t,ps,rh,ice_wat,liq_wat,o3mr] - output variables: [ua,va,t,ps,rh,ice_wat,liq_wat,o3mr] - active variables: [psi,chi,ua,va] - bump: - general: - universe length-scale: 2000.0e3 - io: - data directory: {{atmosphere_bump_data_directory}} - files prefix: psichitouv/psichitouv_gfs - drivers: - read local psichitouv: true -linear variable change: - linear variable change name: Control2Analysis - input variables: [ua,va,t,ps,rh,ice_wat,liq_wat,o3mr] - output variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_gsibec.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_gsibec.yaml.j2 index 73983d706..2b6aa1700 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_gsibec.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_static_gsibec.yaml.j2 @@ -30,7 +30,21 @@ saber outer blocks: ozone_mass_mixing_ratio] linear variable change: linear variable change name: Control2Analysis - input variables: [eastward_wind,northward_wind,air_temperature,air_pressure_at_surface, - water_vapor_mixing_ratio_wrt_moist_air,cloud_liquid_ice,cloud_liquid_water, - mole_fraction_of_ozone_in_air] - output variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] + input variables: + - eastward_wind + - northward_wind + - air_temperature + - air_pressure_at_surface + - water_vapor_mixing_ratio_wrt_moist_air + - cloud_liquid_ice + - cloud_liquid_water + - mole_fraction_of_ozone_in_air + output variables: + - eastward_wind + - northward_wind + - air_temperature + - air_pressure_at_surface + - water_vapor_mixing_ratio_wrt_moist_air + - cloud_liquid_ice + - cloud_liquid_water + - ozone_mass_mixing_ratio diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_cubed_sphere.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_cubed_sphere.yaml.j2 index e49a7daf0..9990e7a4a 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_cubed_sphere.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_cubed_sphere.yaml.j2 @@ -3,7 +3,24 @@ output: filetype: cube sphere history filename: ./anl/cubed_sphere_grid_atminc.jedi.nc provider: ufs - fields to write: [ugrd,vgrd,tmp,pressfc,spfh,icmr,clwmr,o3mr] + fields to write: + - eastward_wind + - northward_wind + - air_temperature + - air_pressure_at_surface + - water_vapor_mixing_ratio_wrt_moist_air + - cloud_liquid_ice + - cloud_liquid_water + - ozone_mass_mixing_ratio + field io names: + eastward_wind: ugrd + northward_wind: vgrd + air_temperature: tmp + air_pressure_at_surface: pressfc + water_vapor_mixing_ratio_wrt_moist_air: spfh + cloud_liquid_ice: icmr + cloud_liquid_water: clwmr + ozone_mass_mixing_ratio: o3mr geometry: fms initialization: namelist filename: "{{atmosphere_fv3jedi_files_path}}/fmsmpp.nml" @@ -15,4 +32,3 @@ geometry: npx: {{atmosphere_npx_ges}} npy: {{atmosphere_npy_ges}} npz: {{atmosphere_npz_ges}} - field metadata override: "{{atmosphere_fv3jedi_files_path}}/fv3jedi_fieldmetadata_history.yaml" diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_gaussian.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_gaussian.yaml.j2 index 1d3bc4c53..414aa0bd6 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_gaussian.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_gaussian.yaml.j2 @@ -3,6 +3,15 @@ output: filetype: auxgrid gridtype: gaussian filename: {{atmosphere_final_increment_prefix}} + field io names: + eastward_wind: ugrd + northward_wind: vgrd + air_temperature: tmp + air_pressure_at_surface: pressfc + water_vapor_mixing_ratio_wrt_moist_air: spfh + cloud_liquid_ice: icmr + cloud_liquid_water: clwmr + ozone_mass_mixing_ratio: o3mr geometry: fms initialization: namelist filename: "{{atmosphere_fv3jedi_files_path}}/fmsmpp.nml" @@ -14,4 +23,3 @@ geometry: npx: {{atmosphere_npx_anl}} npy: {{atmosphere_npy_anl}} npz: {{atmosphere_npz_anl}} - field metadata override: "{{atmosphere_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_geometry_background.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_geometry_background.yaml.j2 index 8e0f501ea..54e847648 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_geometry_background.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_geometry_background.yaml.j2 @@ -8,4 +8,3 @@ layout: npx: {{atmosphere_npx_ges}} npy: {{atmosphere_npy_ges}} npz: {{atmosphere_npz_ges}} -field metadata override: "{{atmosphere_fv3jedi_files_path}}/fv3jedi_fieldmetadata_history.yaml" diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_model_pseudo.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_model_pseudo.yaml.j2 index cf054e369..74af652ba 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_model_pseudo.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_model_pseudo.yaml.j2 @@ -8,6 +8,25 @@ filename_sfcd: '%yyyy%mm%dd.%hh%MM%ss.sfc_data.nc' filename_sfcw: '%yyyy%mm%dd.%hh%MM%ss.fv_srf_wnd.res.nc' filename_cplr: '%yyyy%mm%dd.%hh%MM%ss.coupler.res' tstep: {{atmosphere_forecast_timestep}} -model variables: [ua,va,t,delp,sphum,ice_wat,liq_wat,o3mr,phis, - slmsk,sheleg,tsea,vtype,stype,vfrac,stc,smc,snwdph, - u_srf,v_srf,f10m] +field io names: + eastward_wind: ugrd + northward_wind: vgrd + air_temperature: tmp + layer_thickness: delz + air_pressure_thickness: dpres + air_pressure_at_surface: pressfc + water_vapor_mixing_ratio_wrt_moist_air: spfh + cloud_liquid_ice: icmr + cloud_liquid_water: clwmr + ozone_mass_mixing_ratio: o3mr + geopotential_height_at_surface: hgtsfc + slmsk: land + sheleg: weasd + skin_temperature_at_surface: tmpsfc + stype: sotyp + vfrac: veg + totalSnowDepthMeters: snod + eastward_wind_at_surface: ugrd_hyblev1 + northward_wind_at_surface: vgrd_hyblev1 + soilm: soilw1 + soilt: soilt1 diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_output_ensemble_increments_cubed_sphere.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_output_ensemble_increments_cubed_sphere.yaml.j2 index 8d39113bd..b835948f5 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_output_ensemble_increments_cubed_sphere.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_output_ensemble_increments_cubed_sphere.yaml.j2 @@ -1,3 +1,12 @@ filetype: cube sphere history provider: ufs filename: ./anl/mem%{member}%/cubed_sphere_grid_atminc.jedi.nc +field io names: + eastward_wind: ugrd + northward_wind: vgrd + air_temperature: tmp + air_pressure_at_surface: pressfc + water_vapor_mixing_ratio_wrt_moist_air: spfh + cloud_liquid_ice: icmr + cloud_liquid_water: clwmr + ozone_mass_mixing_ratio: o3mr diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_output_ensemble_increments_gaussian.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_output_ensemble_increments_gaussian.yaml.j2 index fc6674f55..6d3d3b2af 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_output_ensemble_increments_gaussian.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_output_ensemble_increments_gaussian.yaml.j2 @@ -1,3 +1,12 @@ filetype: auxgrid gridtype: gaussian filename: {{atmosphere_ensemble_increment_prefix}} +field io names: + eastward_wind: ugrd + northward_wind: vgrd + air_temperature: tmp + air_pressure_at_surface: pressfc + water_vapor_mixing_ratio_wrt_moist_air: spfh + cloud_liquid_ice: icmr + cloud_liquid_water: clwmr + ozone_mass_mixing_ratio: o3mr diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_posterior_output_gaussian.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_posterior_output_gaussian.yaml.j2 index 512d2350f..a14180d5f 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_posterior_output_gaussian.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_posterior_output_gaussian.yaml.j2 @@ -1,3 +1,12 @@ filetype: auxgrid gridtype: gaussian filename: {{atmosphere_posterior_output_gaussian}} +field io names: + eastward_wind: ugrd + northward_wind: vgrd + air_temperature: tmp + air_pressure_at_surface: pressfc + water_vapor_mixing_ratio_wrt_moist_air: spfh + cloud_liquid_ice: icmr + cloud_liquid_water: clwmr + ozone_mass_mixing_ratio: o3mr diff --git a/parm/jcb-gdas/model/snow/snow_3dvar_outer_loop_1.yaml.j2 b/parm/jcb-gdas/model/snow/snow_3dvar_outer_loop_1.yaml.j2 index 48de31e89..07aacc206 100644 --- a/parm/jcb-gdas/model/snow/snow_3dvar_outer_loop_1.yaml.j2 +++ b/parm/jcb-gdas/model/snow/snow_3dvar_outer_loop_1.yaml.j2 @@ -12,13 +12,15 @@ npx: {{ snow_npx_anl }} npy: {{ snow_npy_anl }} npz: {{ snow_npz_anl }} - field metadata override: {{ snow_fv3jedi_files_path }}/fv3jedi_fieldmetadata_restart.yaml time invariant fields: state fields: datetime: '{{ snow_background_time_iso }}' filetype: fms restart skip coupler file: true - state variables: [orog_filt] + state variables: + - filtered_orography + field io names: + filtered_orography: orog_filt datapath: {{ snow_orog_files_path }}/ filename_orog: {{ snow_orog_prefix }}_oro_data.nc diagnostics: diff --git a/parm/jcb-gdas/model/snow/snow_background.yaml.j2 b/parm/jcb-gdas/model/snow/snow_background.yaml.j2 index b993a3eb8..79630fb94 100644 --- a/parm/jcb-gdas/model/snow/snow_background.yaml.j2 +++ b/parm/jcb-gdas/model/snow/snow_background.yaml.j2 @@ -2,7 +2,17 @@ datapath: {{ snow_background_path }} filetype: fms restart skip coupler file: true datetime: '{{ snow_background_time_iso }}' -state variables: [snodl,vtype,slmsk,sheleg,orog_filt,fice] +state variables: +- totalSnowDepth +- vtype +- slmsk +- sheleg +- filtered_orography +- fraction_of_ice +field io names: + totalSnowDepth: snodl + filtered_orography: orog_filt + fraction_of_ice: fice filename_sfcd: '{{ snow_background_time_fv3 }}.sfc_data.nc' filename_cplr: '{{ snow_background_time_fv3 }}.coupler.res' filename_orog: '{{ snow_orog_prefix }}_oro_data.nc' diff --git a/parm/jcb-gdas/model/snow/snow_final_increment_fms.yaml.j2 b/parm/jcb-gdas/model/snow/snow_final_increment_fms.yaml.j2 index 3bd182f6b..7c9e13e23 100644 --- a/parm/jcb-gdas/model/snow/snow_final_increment_fms.yaml.j2 +++ b/parm/jcb-gdas/model/snow/snow_final_increment_fms.yaml.j2 @@ -5,7 +5,12 @@ output: filetype: fms restart filename_sfcd: '{{ snow_background_time_fv3 }}.sfc_data.nc' filename_cplr: '{{ snow_background_time_fv3 }}.coupler.res' - state variables: [snodl,vtype,slmsk] + state variables: + - totalSnowDepth + - vtype + - slmsk + field io names: + totalSnowDepth: snodl geometry: fms initialization: namelist filename: "{{ snow_fv3jedi_files_path }}/fmsmpp.nml" @@ -17,4 +22,3 @@ geometry: npx: {{ snow_npx_ges }} npy: {{ snow_npy_ges }} npz: {{ snow_npz_ges }} - field metadata override: "{{ snow_fv3jedi_files_path }}/fv3jedi_fieldmetadata_restart.yaml" diff --git a/parm/jcb-gdas/model/snow/snow_geometry_background.yaml.j2 b/parm/jcb-gdas/model/snow/snow_geometry_background.yaml.j2 index e53d5ae35..ff72e6222 100644 --- a/parm/jcb-gdas/model/snow/snow_geometry_background.yaml.j2 +++ b/parm/jcb-gdas/model/snow/snow_geometry_background.yaml.j2 @@ -8,4 +8,3 @@ layout: npx: {{snow_npx_ges}} npy: {{snow_npy_ges}} npz: {{snow_npz_ges}} -field metadata override: "{{snow_fv3jedi_files_path}}/fv3jedi_fieldmetadata_restart.yaml" diff --git a/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml b/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml index 006567743..2b64a1966 100644 --- a/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml +++ b/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml @@ -36,7 +36,7 @@ window_length: PT6H bound_to_include: begin minimizer: DRPCG final_diagnostics_departures: anlmob -analysis_variables: [ua,va,t,ps,sphum,ice_wat,liq_wat,o3mr] +analysis_variables: [eastward_wind,northward_wind,air_temperature,air_pressure_at_surface,water_vapor_mixing_ratio_wrt_moist_air,ice_wat,liq_wat,o3mr] number_of_outer_loops: 2 # Testing From b6b0d7d94c01950af747ab628800588956f0e2d7 Mon Sep 17 00:00:00 2001 From: Guillaume Vernieres Date: Wed, 28 May 2025 09:04:02 -0400 Subject: [PATCH 122/199] Configuration for the new marine sigb (#127) Companion PR of https://github.com/NOAA-EMC/GDASApp/pull/1697 --- .../algorithm/marine/soca_diagb.yaml.j2 | 39 ++++++++-------- .../marine/soca_setcorscales.yaml.j2 | 4 +- .../observations/marine/adt_rads_all.yaml.j2 | 2 +- .../marine/icec_amsr2_north.yaml.j2 | 4 -- .../marine/icec_amsr2_south.yaml.j2 | 4 -- .../marine/insitu_salt_profile_argo.yaml.j2 | 10 ++-- .../marine/insitu_temp_profile_argo.yaml.j2 | 12 ++--- .../observations/marine/sst_generic.yaml.j2 | 46 +++++++++---------- .../marine/sst_generic_geo.yaml.j2 | 46 +++++++++---------- 9 files changed, 76 insertions(+), 91 deletions(-) diff --git a/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 index 17063d565..8369a81af 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 @@ -26,29 +26,26 @@ variables: name: - sea_water_potential_temperature - sea_water_salinity - - eastward_sea_water_velocity - - northward_sea_water_velocity +# - eastward_sea_water_velocity +# - northward_sea_water_velocity - sea_water_cell_thickness - sea_surface_height_above_geoid - sea_ice_area_fraction - sea_ice_thickness - sea_ice_snow_thickness - - mom6_mld - -vertical e-folding scale: 500.0 -min efold depth ratio: 3.0 # keep the e-folding scale at most - # 1/(min efold depth ratio) of the local depth -rescale: 1.0 # rescales the filtered std. dev. by "rescale" -max ssh: 0.0 # Limits the amplitude of the unbalanced bkg err -min depth: 50.0 # zero out the bkg. error. at less than min depth -number of halo points: 4 -number of neighbors: 16 - -simple smoothing: - horizontal iterations: 100 - vertical iterations: 1 - -# TODO(G): Too slow for the below scale -#diffusion: -# horizontal: 500.0e3 -# vertical: 3.0 +# - mom6_mld + + +vertical e-folding scale dynamic: 200.0 +vertical e-folding scale static: 300.0 +stencil growth iterations: 100 +min efold depth ratio: 3.0 +rescale dynamic: 0.2 +rescale static: 1.0 +static sig B: + sigT: 0.5 + sigS: 0.1 + sigSic: 0.01 +max ssh: 0.0 +min depth: 100.0 +vertical bin size: 1.0 diff --git a/parm/jcb-gdas/algorithm/marine/soca_setcorscales.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_setcorscales.yaml.j2 index a30b68588..d87f3371f 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_setcorscales.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_setcorscales.yaml.j2 @@ -16,8 +16,8 @@ islands: scales: vert layers: 5 # in units of layer sea_surface_height_above_geoid: - rossby mult: 1.50 - min grid mult: 2.0 + rossby mult: 1.0 + min grid mult: 1.0 rh output: datadir: ./ diff --git a/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 b/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 index c7e3a1004..dca0983ea 100644 --- a/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 @@ -41,7 +41,7 @@ variables: [GeoVaLs/mesoscale_representation_error, ObsError/absoluteDynamicTopography] coefs: [0.2, - 10.0] + 1.0] - filter: Domain Check where: - variable: { name: GeoVaLs/sea_ice_area_fraction} diff --git a/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 index e1a92201d..f28209243 100644 --- a/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 @@ -39,10 +39,6 @@ action: name: inflate error inflation factor: 2.0 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 100e3 # - filter: Gaussian Thinning # horizontal_mesh: 25 #km # use_reduced_horizontal_grid: true diff --git a/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 index e1a92201d..f28209243 100644 --- a/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 @@ -39,10 +39,6 @@ action: name: inflate error inflation factor: 2.0 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 100e3 # - filter: Gaussian Thinning # horizontal_mesh: 25 #km # use_reduced_horizontal_grid: true diff --git a/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 index b5772500e..9996a2f76 100644 --- a/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 @@ -46,7 +46,7 @@ #----------------------------------------------------------------------------- ### Global range test #----------------------------------------------------------------------------- - + - filter: Bounds Check filter variables: [{name: salinity}] minvalue: 2.0 @@ -125,7 +125,7 @@ #### Northwestern shelves #----------------------------------------------------------------------------- - + - filter: Bounds Check filter variables: [{name: salinity}] minvalue: 0.0 @@ -142,7 +142,7 @@ #### Southwestern shelves #----------------------------------------------------------------------------- - + - filter: Bounds Check filter variables: [{name: salinity}] minvalue: 0.0 @@ -159,7 +159,7 @@ #### Arctic Ocean #----------------------------------------------------------------------------- - + - filter: Bounds Check filter variables: [{name: salinity}] minvalue: 2.0 @@ -183,7 +183,7 @@ variables: - ObsError/salinity coefs: - - 1000.0 + - 100.0 {% if letkf_app | default(false) %} obs localizations: - localization method: Rossby diff --git a/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 index ebd55d2ba..ad401951a 100644 --- a/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 @@ -155,7 +155,7 @@ #### Arctic Ocean #----------------------------------------------------------------------------- - + - filter: Bounds Check filter variables: [{name: waterTemperature}] minvalue: -1.92 @@ -165,10 +165,10 @@ name: MetaData/latitude minvalue: 60 - - filter: Background Check - filter variables: [{name: waterTemperature}] - threshold: 5.0 - absolute threshold: 5.0 +# - filter: Background Check +# filter variables: [{name: waterTemperature}] +# threshold: 5.0 +# absolute threshold: 5.0 - filter: Perform Action action: @@ -179,7 +179,7 @@ variables: - ObsError/waterTemperature coefs: - - 1000.0 + - 100.0 {% if letkf_app | default(false) %} obs localizations: - localization method: Rossby diff --git a/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 index 8c02ef1ad..6f4e9bc9a 100644 --- a/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 @@ -19,7 +19,23 @@ obs error: covariance model: diagonal - obs pre filters: + obs prior filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/distance_from_coast} + minvalue: 50e3 + - filter: Perform Action # accept obs that failed previous QC in Pamlico Sound & Chesapeake Bay + action: + name: accept + where: + - variable: + name: MetaData/latitude + minvalue: 34.5 + maxvalue: 39.7 + - variable: + name: MetaData/longitude + minvalue: -76.88 + maxvalue: -74.5 - filter: Bounds Check minvalue: -1.2 maxvalue: 41.0 @@ -27,8 +43,6 @@ where: - variable: {name: ObsError/seaSurfaceTemperature} minvalue: 1.0e-8 - - obs prior filters: - filter: Domain Check where: - variable: {name: GeoVaLs/sea_area_fraction} @@ -41,10 +55,11 @@ where: - variable: {name: GeoVaLs/sea_surface_temperature} minvalue: -1.2 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 50e3 + + obs post filters: + - filter: Background Check + absolute threshold: 5.0 + {% if letkf_app | default(false) %} obs localizations: - localization method: Rossby @@ -54,20 +69,3 @@ min value: 200.0e3 max value: 900.0e3 {% endif %} - - obs post filters: - - filter: Perform Action # Pamlico Sound & Chesapeake Bay - action: - name: accept - where: - - variable: - name: MetaData/latitude - minvalue: 34.5 - maxvalue: 39.7 - - variable: - name: MetaData/longitude - minvalue: -76.88 - maxvalue: -74.5 -# TODO: var fails when bkg chk filter is last -# - filter: Background Check -# absolute threshold: 5.0 diff --git a/parm/jcb-gdas/observations/marine/sst_generic_geo.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_generic_geo.yaml.j2 index 35a7d2813..642fc7799 100644 --- a/parm/jcb-gdas/observations/marine/sst_generic_geo.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sst_generic_geo.yaml.j2 @@ -19,7 +19,23 @@ obs error: covariance model: diagonal - obs pre filters: + obs prior filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/distance_from_coast} + minvalue: 50e3 + - filter: Perform Action # accept obs that failed previous QC in Pamlico Sound & Chesapeake Bay + action: + name: accept + where: + - variable: + name: MetaData/latitude + minvalue: 34.5 + maxvalue: 39.7 + - variable: + name: MetaData/longitude + minvalue: -76.88 + maxvalue: -74.5 - filter: Bounds Check minvalue: -1.2 maxvalue: 41.0 @@ -27,8 +43,6 @@ where: - variable: {name: ObsError/seaSurfaceTemperature} minvalue: 1.0e-8 - - obs prior filters: - filter: Domain Check where: - variable: {name: GeoVaLs/sea_area_fraction} @@ -41,16 +55,17 @@ where: - variable: {name: GeoVaLs/sea_surface_temperature} minvalue: -1.2 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 50e3 - filter: Domain Check action: name: passivate where: - variable: {name: GeoVaLs/sea_surface_temperature} maxvalue: -4.0 + + obs post filters: + - filter: Background Check + absolute threshold: 5.0 + {% if letkf_app | default(false) %} obs localizations: - localization method: Rossby @@ -60,20 +75,3 @@ min value: 200.0e3 max value: 900.0e3 {% endif %} - - obs post filters: - - filter: Perform Action # Pamlico Sound & Chesapeake Bay - action: - name: accept - where: - - variable: - name: MetaData/latitude - minvalue: 34.5 - maxvalue: 39.7 - - variable: - name: MetaData/longitude - minvalue: -76.88 - maxvalue: -74.5 -# TODO: var fails when bkg chk filter is last -# - filter: Background Check -# absolute threshold: 5.0 From 135f1e80e303c9c5a5a5a65f6f8da8293f84c392 Mon Sep 17 00:00:00 2001 From: Guillaume Vernieres Date: Wed, 28 May 2025 14:37:06 -0400 Subject: [PATCH 123/199] Configuration for the simple incr bound check (#128) --- .../algorithm/marine/socaincr2mom6.yaml.j2 | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 b/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 index 4218766be..d1d044c20 100644 --- a/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 @@ -37,3 +37,22 @@ output increment: exp: mom6_iau type: incr output file: inc.nc + +qc increment: + background: + date: '{{ marine_window_begin_iso }}' + basename: ./INPUT/ + ocn_filename: MOM.res.nc + read_from_file: 1 + state variables: + - sea_water_potential_temperature + - sea_water_salinity + - sea_surface_height_above_geoid + - eastward_sea_water_velocity + - northward_sea_water_velocity + - sea_water_cell_thickness + state bounds: + sea_water_potential_temperature: [-2.5, 36.0] + sea_water_salinity: [0.0, 44.0] + increment max: + steric: 0.5 From 592eb18662d2b55f649cfc8e7bf9f02d3e97f950 Mon Sep 17 00:00:00 2001 From: yuanxue2870 <136842224+yuanxue2870@users.noreply.github.com> Date: Mon, 2 Jun 2025 11:15:42 -0400 Subject: [PATCH 124/199] Update madis_snow YAML and add reject list (#132) The PR directly solves issue: https://github.com/NOAA-EMC/jcb-gdas/issues/131 The intent is to make the madis_snow yaml consistent with other (current as of 05/30/2025) yamls. A reject list is also added to include 77 bad madis_snow stations identified between 2021 and 2023. G-W tests are conducted to make sure the newly-added madis_snow reject list is read by JEDI. --- .../snow/t00z/madis_snow.yaml | 2 +- .../snow/t06z/madis_snow.yaml | 2 +- .../observations/snow/madis_snow.yaml.j2 | 64 ++++++++++++++++++- 3 files changed, 65 insertions(+), 3 deletions(-) diff --git a/parm/jcb-gdas/observation_chronicle/snow/t00z/madis_snow.yaml b/parm/jcb-gdas/observation_chronicle/snow/t00z/madis_snow.yaml index 1e9fd35e6..3171740d6 100644 --- a/parm/jcb-gdas/observation_chronicle/snow/t00z/madis_snow.yaml +++ b/parm/jcb-gdas/observation_chronicle/snow/t00z/madis_snow.yaml @@ -9,7 +9,7 @@ window_option: max # observation type initial configuration # -------------------------------- -stations_to_reject: [fakelist] +stations_to_reject: [BF001, BF002, BF004, BF009, BF010, BF012, BF017, BF019, BF030, BF034, BF035, BF036, BF037, BF038, BF044, BF049, BF052, BF054, BF056, BF059, BF061, BF066, BF069, BF078, BF080, BF084, BF086, BF089, BF091, BF110, BF117, BF121, BF122, BF123, BF132, BF137, BF142, BF143, BF150, BF151, BF159, BF191, BF197, BF209, BF215, BF233, BF234, BF235, BF242, BF284, BOWC1, BT007, BT008, BT116, BT117, BT119, BT128, BT140, CAAXM, CAINP, CALXV, CSNA2, FAFA2, HPN06, KESQ6, LACQ6, LPTUT, MKLA2, OHAW1, OJIM4, OKEQ6, SHDA2, TUNA2, UTRMD, WEAQ6, WINM4, XACC1] # Chronicle of changes for this observation type # ---------------------------------------- diff --git a/parm/jcb-gdas/observation_chronicle/snow/t06z/madis_snow.yaml b/parm/jcb-gdas/observation_chronicle/snow/t06z/madis_snow.yaml index 0fed1104d..06a7560b2 100644 --- a/parm/jcb-gdas/observation_chronicle/snow/t06z/madis_snow.yaml +++ b/parm/jcb-gdas/observation_chronicle/snow/t06z/madis_snow.yaml @@ -9,7 +9,7 @@ window_option: max # observation type initial configuration # -------------------------------- -stations_to_reject: [fakelist] +stations_to_reject: [BF001, BF002, BF004, BF009, BF010, BF012, BF017, BF019, BF030, BF034, BF035, BF036, BF037, BF038, BF044, BF049, BF052, BF054, BF056, BF059, BF061, BF066, BF069, BF078, BF080, BF084, BF086, BF089, BF091, BF110, BF117, BF121, BF122, BF123, BF132, BF137, BF142, BF143, BF150, BF151, BF159, BF191, BF197, BF209, BF215, BF233, BF234, BF235, BF242, BF284, BOWC1, BT007, BT008, BT116, BT117, BT119, BT128, BT140, CAAXM, CAINP, CALXV, CSNA2, FAFA2, HPN06, KESQ6, LACQ6, LPTUT, MKLA2, OHAW1, OJIM4, OKEQ6, SHDA2, TUNA2, UTRMD, WEAQ6, WINM4, XACC1] # Chronicle of changes for this observation type # ---------------------------------------- diff --git a/parm/jcb-gdas/observations/snow/madis_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/madis_snow.yaml.j2 index f13df6303..c0120105d 100644 --- a/parm/jcb-gdas/observations/snow/madis_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/madis_snow.yaml.j2 @@ -50,6 +50,8 @@ initial value: false - name: elevation_bkgdiff initial value: false + - name: rejectlist + initial value: false - name: background_check initial value: false - name: buddy_check @@ -131,6 +133,11 @@ flag: invalid_elevation ignore: rejected observations - name: reject + - filter: Domain Check # land only, no sea ice + where: + - variable: + name: GeoVaLs/fraction_of_ice + maxvalue: 0.0 - filter: RejectList # no land-ice where: - variable: @@ -145,17 +152,72 @@ - filter: Difference Check # elevation check reference: MetaData/stationElevation value: GeoVaLs/filtered_orography - threshold: 200. + threshold: 800. actions: - name: set flag: elevation_bkgdiff ignore: rejected observations - name: reject + # Add inflate error characterized by a function of elevation difference + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/elevation_diff + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: [MetaData/stationElevation, GeoVaLs/filtered_orography] + coefficients: [1., -1.] + total exponent: 2 + total coefficient: 0.0000015625 #1/(800*800) + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/elevation_diff_exp + type: float + function: + name: ObsFunction/Exponential + options: + variables: [DerivedMetaData/elevation_diff] + coeffA: 1. + coeffB: -1. + coeffC: 0. + coeffD: 1. + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/elevation_diff_exp_inv + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: [DerivedMetaData/elevation_diff_exp] + total exponent: -1. + - filter: Perform Action + filter variables: + - name: totalSnowDepth + where: + - variable: + name: ObsValue/totalSnowDepth + minvalue: 0. + action: + name: inflate error + inflation variable: DerivedMetaData/elevation_diff_exp_inv # 2.0 + - filter: BlackList + where: + - variable: + name: MetaData/stationIdentification + is_in: {{ get_conventional_rejected_stations(observation_from_jcb) }} + actions: + - name: set + flag: rejectlist + ignore: rejected observations + - name: reject obs post filters: - filter: Background Check # gross error check filter variables: - name: totalSnowDepth threshold: 6.25 + absolute threshold: 250 + bias correction parameter: 1.0 actions: - name: set flag: background_check From 35a2159572af36d12db015f8700738746308600e Mon Sep 17 00:00:00 2001 From: yuanxue2870 <136842224+yuanxue2870@users.noreply.github.com> Date: Mon, 2 Jun 2025 13:44:47 -0400 Subject: [PATCH 125/199] Add snocvr rejectlist (#130) The PR directly solves the issue: https://github.com/NOAA-EMC/jcb-gdas/issues/129 Tests will be done in the G-W by Jiarui @jiaruidong2017 either today or tomorrow. Upon submitting this PR, no G-W test has been done yet due to time limit. The PR is ready for review, but since I am awaiting the test output. The PR will be in the draft mode until we get a successful run. Update (06/02/2025): new re-run at /scratch2/NCEPDEV/stmp1/Youlong.Xia/snow2dvar/COMROOT/C384mx025_2dvar/logs/2024091100/gdas_snowanl.log shows snowcvr reject list has been read by JEDI correctly. --------- Co-authored-by: Jiarui Dong --- .../observation_chronicle/snow/t00z/snocvr.yaml | 2 +- .../observation_chronicle/snow/t06z/snocvr.yaml | 2 +- parm/jcb-gdas/observations/snow/snocvr.yaml.j2 | 12 ++++++++++++ 3 files changed, 14 insertions(+), 2 deletions(-) diff --git a/parm/jcb-gdas/observation_chronicle/snow/t00z/snocvr.yaml b/parm/jcb-gdas/observation_chronicle/snow/t00z/snocvr.yaml index 102dc2c3a..7d0fde292 100644 --- a/parm/jcb-gdas/observation_chronicle/snow/t00z/snocvr.yaml +++ b/parm/jcb-gdas/observation_chronicle/snow/t00z/snocvr.yaml @@ -8,7 +8,7 @@ window_option: max # observation type initial configuration # -------------------------------- -stations_to_reject: [fakelist] +stations_to_reject: [BFTU1, FRDN5, MWDO3, SGNN5, NUTA3, CNDA3, HBEA3, ELCN5] # Chronicle of changes for this observation type # ---------------------------------------- diff --git a/parm/jcb-gdas/observation_chronicle/snow/t06z/snocvr.yaml b/parm/jcb-gdas/observation_chronicle/snow/t06z/snocvr.yaml index 06f123c16..222770c84 100644 --- a/parm/jcb-gdas/observation_chronicle/snow/t06z/snocvr.yaml +++ b/parm/jcb-gdas/observation_chronicle/snow/t06z/snocvr.yaml @@ -8,7 +8,7 @@ window_option: max # observation type initial configuration # -------------------------------- -stations_to_reject: [fakelist] +stations_to_reject: [BFTU1, FRDN5, MWDO3, SGNN5, NUTA3, CNDA3, HBEA3, ELCN5] # Chronicle of changes for this observation type # ---------------------------------------- diff --git a/parm/jcb-gdas/observations/snow/snocvr.yaml.j2 b/parm/jcb-gdas/observations/snow/snocvr.yaml.j2 index c45029125..d5ac5b6af 100644 --- a/parm/jcb-gdas/observations/snow/snocvr.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/snocvr.yaml.j2 @@ -51,6 +51,8 @@ initial value: false - name: elevation_bkgdiff initial value: false + - name: rejectlist + initial value: false - name: background_check initial value: false - name: buddy_check @@ -200,6 +202,16 @@ action: name: inflate error inflation variable: DerivedMetaData/elevation_diff_exp_inv # 2.0 + - filter: BlackList + where: + - variable: + name: MetaData/stationIdentification + is_in: {{ get_conventional_rejected_stations(observation_from_jcb) }} + actions: + - name: set + flag: rejectlist + ignore: rejected observations + - name: reject obs post filters: - filter: Background Check # gross error check filter variables: From ec65ef5676c724f9f67a157f1c8744254dc5fee6 Mon Sep 17 00:00:00 2001 From: Anna Shlyaeva Date: Mon, 9 Jun 2025 15:11:26 -0600 Subject: [PATCH 126/199] Bugfix for marine obs over model's land (#136) Occasionally, marine observations are surrounded by what model thinks is land. In this case the check for obs over land needs to check whether the mask is defined in addition to checking the mask values. This is fixed here. Closes https://github.com/NOAA-EMC/GDASApp/issues/1739 Co-authored-by: Anna Shlyaeva --- parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/icec_atms_n20_l2.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/icec_atms_n21_l2.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/icec_atms_npp_l2.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 | 1 + .../jcb-gdas/observations/marine/icec_viirs_n20_l2_north.yaml.j2 | 1 + .../jcb-gdas/observations/marine/icec_viirs_n20_l2_south.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/insitu_profile_argo.yaml.j2 | 1 + .../observations/marine/insitu_salt_profile_argo.yaml.j2 | 1 + .../observations/marine/insitu_temp_profile_argo.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/sss_smap_l2.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/sss_smos_l2.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/sst_generic_geo.yaml.j2 | 1 + 19 files changed, 19 insertions(+) diff --git a/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 b/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 index dca0983ea..9c75aa5e6 100644 --- a/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 @@ -21,6 +21,7 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/sea_area_fraction} + value: is_valid minvalue: 0.9 - filter: Domain Check where: diff --git a/parm/jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 index 5fd44160e..95c3fe345 100644 --- a/parm/jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 @@ -22,6 +22,7 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/sea_area_fraction} + value: is_valid minvalue: 0.9 - filter: Bounds Check minvalue: 0.0 diff --git a/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 index f28209243..50770cf00 100644 --- a/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 @@ -22,6 +22,7 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/sea_area_fraction} + value: is_valid minvalue: 0.9 - filter: Bounds Check minvalue: 0.0 diff --git a/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 index f28209243..50770cf00 100644 --- a/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 @@ -22,6 +22,7 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/sea_area_fraction} + value: is_valid minvalue: 0.9 - filter: Bounds Check minvalue: 0.0 diff --git a/parm/jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 index 5fd44160e..95c3fe345 100644 --- a/parm/jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 @@ -22,6 +22,7 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/sea_area_fraction} + value: is_valid minvalue: 0.9 - filter: Bounds Check minvalue: 0.0 diff --git a/parm/jcb-gdas/observations/marine/icec_atms_n20_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_atms_n20_l2.yaml.j2 index 5fd44160e..95c3fe345 100644 --- a/parm/jcb-gdas/observations/marine/icec_atms_n20_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_atms_n20_l2.yaml.j2 @@ -22,6 +22,7 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/sea_area_fraction} + value: is_valid minvalue: 0.9 - filter: Bounds Check minvalue: 0.0 diff --git a/parm/jcb-gdas/observations/marine/icec_atms_n21_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_atms_n21_l2.yaml.j2 index 5fd44160e..95c3fe345 100644 --- a/parm/jcb-gdas/observations/marine/icec_atms_n21_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_atms_n21_l2.yaml.j2 @@ -22,6 +22,7 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/sea_area_fraction} + value: is_valid minvalue: 0.9 - filter: Bounds Check minvalue: 0.0 diff --git a/parm/jcb-gdas/observations/marine/icec_atms_npp_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_atms_npp_l2.yaml.j2 index 5fd44160e..95c3fe345 100644 --- a/parm/jcb-gdas/observations/marine/icec_atms_npp_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_atms_npp_l2.yaml.j2 @@ -22,6 +22,7 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/sea_area_fraction} + value: is_valid minvalue: 0.9 - filter: Bounds Check minvalue: 0.0 diff --git a/parm/jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 index 946c2d42a..ed60b17d3 100644 --- a/parm/jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 @@ -22,6 +22,7 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/sea_area_fraction} + value: is_valid minvalue: 0.9 - filter: Bounds Check minvalue: 0.0 diff --git a/parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 index 5fd44160e..95c3fe345 100644 --- a/parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 @@ -22,6 +22,7 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/sea_area_fraction} + value: is_valid minvalue: 0.9 - filter: Bounds Check minvalue: 0.0 diff --git a/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_north.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_north.yaml.j2 index 5fd44160e..95c3fe345 100644 --- a/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_north.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_north.yaml.j2 @@ -22,6 +22,7 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/sea_area_fraction} + value: is_valid minvalue: 0.9 - filter: Bounds Check minvalue: 0.0 diff --git a/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_south.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_south.yaml.j2 index 5fd44160e..95c3fe345 100644 --- a/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_south.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_south.yaml.j2 @@ -22,6 +22,7 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/sea_area_fraction} + value: is_valid minvalue: 0.9 - filter: Bounds Check minvalue: 0.0 diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_argo.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_argo.yaml.j2 index f26cd42f0..2b08a67bc 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_argo.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_argo.yaml.j2 @@ -46,6 +46,7 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/sea_area_fraction} + value: is_valid minvalue: 0.5 #------------------------------------------------------------------------------- diff --git a/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 index 9996a2f76..e7e0c0463 100644 --- a/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 @@ -38,6 +38,7 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/sea_area_fraction} + value: is_valid minvalue: 0.5 diff --git a/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 index ad401951a..8edb0b136 100644 --- a/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 @@ -34,6 +34,7 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/sea_area_fraction} + value: is_valid minvalue: 0.5 #------------------------------------------------------------------------------- diff --git a/parm/jcb-gdas/observations/marine/sss_smap_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/sss_smap_l2.yaml.j2 index 6dd8d64cf..bdca422dd 100644 --- a/parm/jcb-gdas/observations/marine/sss_smap_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sss_smap_l2.yaml.j2 @@ -22,6 +22,7 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/sea_area_fraction} + value: is_valid minvalue: 0.9 - filter: Bounds Check minvalue: 0.1 diff --git a/parm/jcb-gdas/observations/marine/sss_smos_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/sss_smos_l2.yaml.j2 index 6dd8d64cf..bdca422dd 100644 --- a/parm/jcb-gdas/observations/marine/sss_smos_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sss_smos_l2.yaml.j2 @@ -22,6 +22,7 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/sea_area_fraction} + value: is_valid minvalue: 0.9 - filter: Bounds Check minvalue: 0.1 diff --git a/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 index 6f4e9bc9a..02cd72106 100644 --- a/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 @@ -46,6 +46,7 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/sea_area_fraction} + value: is_valid minvalue: 0.9 - filter: Domain Check where: diff --git a/parm/jcb-gdas/observations/marine/sst_generic_geo.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_generic_geo.yaml.j2 index 642fc7799..68e2b7349 100644 --- a/parm/jcb-gdas/observations/marine/sst_generic_geo.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sst_generic_geo.yaml.j2 @@ -46,6 +46,7 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/sea_area_fraction} + value: is_valid minvalue: 0.9 - filter: Domain Check where: From eede23b970aa7936c959710469ffdfdae17a6369 Mon Sep 17 00:00:00 2001 From: Guillaume Vernieres Date: Wed, 11 Jun 2025 15:39:16 +0000 Subject: [PATCH 127/199] New config for stability check (#134) --- parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 b/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 index d1d044c20..a83dcb63b 100644 --- a/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 @@ -51,8 +51,15 @@ qc increment: - eastward_sea_water_velocity - northward_sea_water_velocity - sea_water_cell_thickness + - ocean_mixed_layer_thickness + - sea_water_depth state bounds: sea_water_potential_temperature: [-2.5, 36.0] sea_water_salinity: [0.0, 44.0] increment max: steric: 0.5 + increment stability iterations: 20 + min stable density gradient: 1e-4 + steric increment: + linear variable changes: + - linear variable change name: BalanceSOCA From 96da98e3fdd2b9622a9b05e2cb4d49ce2abe866b Mon Sep 17 00:00:00 2001 From: Anna Shlyaeva Date: Thu, 12 Jun 2025 08:16:19 -0600 Subject: [PATCH 128/199] Combine soca increment postprocessing in one step (#135) Combines two yamls that are used in soca postprocessing into one (based on `socaincr2mom6.yaml.j2` with soca2cice additions from `soca_2cice_global.yaml.j2` Needs to be merged with https://github.com/NOAA-EMC/GDASApp/pull/1729 and gw PR, otherwise tests will be broken. Co-authored-by: Anna Shlyaeva --- .../marine/soca_2cice_global.yaml.j2 | 68 ------------ .../algorithm/marine/soca_incpostproc.yaml.j2 | 105 ++++++++++++++++++ .../algorithm/marine/socaincr2mom6.yaml.j2 | 65 ----------- 3 files changed, 105 insertions(+), 133 deletions(-) delete mode 100644 parm/jcb-gdas/algorithm/marine/soca_2cice_global.yaml.j2 create mode 100644 parm/jcb-gdas/algorithm/marine/soca_incpostproc.yaml.j2 delete mode 100644 parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/marine/soca_2cice_global.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_2cice_global.yaml.j2 deleted file mode 100644 index 25be1a7aa..000000000 --- a/parm/jcb-gdas/algorithm/marine/soca_2cice_global.yaml.j2 +++ /dev/null @@ -1,68 +0,0 @@ -input geometry: - mom6_input_nml: mom_input.nml - fields metadata: fields_metadata.yaml - -output geometry: - mom6_input_nml: mom_input.nml - fields metadata: fields_metadata.yaml - -variable change: - variable change name: Soca2Cice - arctic: - seaice edge: 0.4 - shuffle: true - rescale prior: - rescale: true - min hice: 0.5 - min hsno: 0.1 - antarctic: - seaice edge: 0.4 - shuffle: true - rescale prior: - rescale: true - min hice: 0.5 - min hsno: 0.1 - cice background state: - restart: ./Data/{{ marine_cice_date_fms }}.cice_model.res.nc - ncat: 5 - ice_lev: 7 - sno_lev: 1 - cice output: - restart: ./Data/{{ marine_cice_date_fms }}.cice_model.res.nc - increment output: - datadir: Data - exp: soca2cice - type: incr - date: {{ marine_cice_date_iso }} - soca increment: - read_from_file: 1 - basename: ./ - ocn_filename: ./Data/ocn.3dvarfgat_pseudo.incr.{{ marine_window_middle_iso }}.nc - ice_filename: ./Data/ice.3dvarfgat_pseudo.incr.{{ marine_window_middle_iso }}.nc - date: {{ marine_cice_date_iso }} - output variables: - - sea_water_potential_temperature - - sea_water_salinity - - sea_ice_area_fraction - - sea_ice_thickness - - sea_ice_snow_thickness - -states: -- input: - read_from_file: 1 - basename: ./ - ocn_filename: ./Data/ocn.3dvarfgat_pseudo.an.{{ marine_window_middle_iso }}.nc - ice_filename: ./Data/ice.3dvarfgat_pseudo.an.{{ marine_window_middle_iso }}.nc - date: {{ marine_cice_date_iso }} - state variables: - - sea_water_potential_temperature - - sea_water_salinity - - sea_ice_area_fraction - - sea_ice_thickness - - sea_ice_snow_thickness - - output: - datadir: Data - exp: soca2cice - type: fc - date: {{ marine_cice_date_iso }} diff --git a/parm/jcb-gdas/algorithm/marine/soca_incpostproc.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_incpostproc.yaml.j2 new file mode 100644 index 000000000..613820cf8 --- /dev/null +++ b/parm/jcb-gdas/algorithm/marine/soca_incpostproc.yaml.j2 @@ -0,0 +1,105 @@ +geometry: + mom6_input_nml: mom_input.nml + fields metadata: ./fields_metadata.yaml + +date: '{{ marine_window_begin_iso }}' + +layers variable: [sea_water_cell_thickness] + +domains: [ocn, ice] + +increment variables: +- sea_water_potential_temperature +- sea_water_salinity +#- eastward_sea_water_velocity +#- northward_sea_water_velocity +- sea_surface_height_above_geoid # Not used in MOM6 iau +- sea_ice_area_fraction +- sea_ice_thickness +- sea_ice_snow_thickness + +set increment variables to zero: +- eastward_sea_water_velocity +- northward_sea_water_velocity + +vertical geometry: + date: '{{ marine_window_begin_iso }}' + basename: ./INPUT/ + ocn_filename: MOM.res.nc + read_from_file: 1 + +soca increment: + date: '{{ marine_window_begin_iso }}' + basename: ./Data/ + ocn_filename: 'ocn.3dvarfgat_pseudo.incr.{{ marine_window_middle_iso }}.nc' + ice_filename: 'ice.3dvarfgat_pseudo.incr.{{ marine_window_middle_iso }}.nc' + read_from_file: 1 + +output increment: + datadir: ./ + date: '{{ marine_window_begin_iso }}' + exp: mom6_iau + type: incr + output file: inc.nc + +qc increment: + state bounds: + sea_water_potential_temperature: [-2.5, 36.0] + sea_water_salinity: [0.0, 44.0] + increment max: + steric: 0.5 + increment stability iterations: 20 + min stable density gradient: 1e-4 + steric increment: + linear variable changes: + - linear variable change name: BalanceSOCA + +soca background: # background at the beginning of the window, used to compute analysis at the beginning of the window for soca2cice + date: '{{ marine_window_begin_iso }}' + basename: ./INPUT/ + ocn_filename: MOM.res.nc + ice_filename: cice.res.nc + read_from_file: 1 + state variables: + - sea_water_potential_temperature + - sea_water_salinity + - sea_surface_height_above_geoid + - eastward_sea_water_velocity + - northward_sea_water_velocity + - ocean_mixed_layer_thickness + - sea_water_depth + - sea_water_cell_thickness + - sea_ice_area_fraction + - sea_ice_thickness + - sea_ice_snow_thickness + +ice analysis postprocessing: + variable change name: Soca2Cice + arctic: + seaice edge: 0.4 + shuffle: true + rescale prior: + rescale: true + min hice: 0.5 + min hsno: 0.1 + antarctic: + seaice edge: 0.4 + shuffle: true + rescale prior: + rescale: true + min hice: 0.5 + min hsno: 0.1 + cice background state: + restart: ./Data/{{ marine_cice_date_fms }}.cice_model.res.nc + ncat: 5 + ice_lev: 7 + sno_lev: 1 + cice output: + restart: ./Data/{{ marine_cice_date_fms }}.cice_model.res.nc + output variables: + - sea_water_potential_temperature + - sea_water_salinity + - sea_ice_area_fraction + - sea_ice_thickness + - sea_ice_snow_thickness + diff --git a/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 b/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 deleted file mode 100644 index a83dcb63b..000000000 --- a/parm/jcb-gdas/algorithm/marine/socaincr2mom6.yaml.j2 +++ /dev/null @@ -1,65 +0,0 @@ -geometry: - mom6_input_nml: mom_input.nml - fields metadata: ./fields_metadata.yaml - -date: '{{ marine_window_begin_iso }}' - -layers variable: [sea_water_cell_thickness] - -domains: [ocn] - -increment variables: -- sea_water_potential_temperature -- sea_water_salinity -#- eastward_sea_water_velocity -#- northward_sea_water_velocity -- sea_surface_height_above_geoid # Not used in MOM6 iau - -set increment variables to zero: -- eastward_sea_water_velocity -- northward_sea_water_velocity - -vertical geometry: - date: '{{ marine_window_begin_iso }}' - basename: ./INPUT/ - ocn_filename: MOM.res.nc - read_from_file: 1 - -soca increment: - date: '{{ marine_window_begin_iso }}' - basename: ./Data/ - ocn_filename: 'ocn.3dvarfgat_pseudo.incr.{{ marine_window_middle_iso }}.nc' - read_from_file: 1 - -output increment: - datadir: ./ - date: '{{ marine_window_begin_iso }}' - exp: mom6_iau - type: incr - output file: inc.nc - -qc increment: - background: - date: '{{ marine_window_begin_iso }}' - basename: ./INPUT/ - ocn_filename: MOM.res.nc - read_from_file: 1 - state variables: - - sea_water_potential_temperature - - sea_water_salinity - - sea_surface_height_above_geoid - - eastward_sea_water_velocity - - northward_sea_water_velocity - - sea_water_cell_thickness - - ocean_mixed_layer_thickness - - sea_water_depth - state bounds: - sea_water_potential_temperature: [-2.5, 36.0] - sea_water_salinity: [0.0, 44.0] - increment max: - steric: 0.5 - increment stability iterations: 20 - min stable density gradient: 1e-4 - steric increment: - linear variable changes: - - linear variable change name: BalanceSOCA From b46c88fc0c9cdcfc84d14d83c5f402290919c5ec Mon Sep 17 00:00:00 2001 From: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com> Date: Thu, 12 Jun 2025 10:40:35 -0400 Subject: [PATCH 129/199] add max allowable geometry difference to atmospheric templates (#138) This PR adds `max allowable geometry difference: 1e-4` to select templates in `algorithm/atmosphere` and `model/atmosphere`. This changes are required by fv3-jedi PR [#1356](https://github.com/JCSDA-internal/fv3-jedi/pull/1356). Co-authored-by: Dan Holdaway <27729500+danholdaway@users.noreply.github.com> --- parm/jcb-gdas/algorithm/atmosphere/atm_addincrement.yaml.j2 | 1 + .../algorithm/atmosphere/fv3jedi_correction_increment.yaml.j2 | 1 + .../algorithm/atmosphere/fv3jedi_ensemble_recenter.yaml.j2 | 1 + parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 | 1 + .../algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 | 1 + parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 | 1 + .../model/atmosphere/atmosphere_background_ensemble.yaml.j2 | 1 + .../atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 | 1 + 8 files changed, 8 insertions(+) diff --git a/parm/jcb-gdas/algorithm/atmosphere/atm_addincrement.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/atm_addincrement.yaml.j2 index e2fcccec8..4d6a8f410 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/atm_addincrement.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/atm_addincrement.yaml.j2 @@ -47,6 +47,7 @@ state: provider: ufs filename: {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_atmf006.nc datapath: ./ + max allowable geometry difference: 1e-4 increment: added variables: - eastward_wind diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_correction_increment.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_correction_increment.yaml.j2 index 95eab9c28..200897491 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_correction_increment.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_correction_increment.yaml.j2 @@ -60,6 +60,7 @@ forecast hours: provider: ufs datapath: ./ filename: {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_atmf{{ '%03d' % atmosphere_iau_hours[ihour] }}.nc + max allowable geometry difference: 1e-04 field io names: eastward_wind: ugrd northward_wind: vgrd diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_ensemble_recenter.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_ensemble_recenter.yaml.j2 index abbdaf140..baa6a005a 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_ensemble_recenter.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_ensemble_recenter.yaml.j2 @@ -49,6 +49,7 @@ members: provider: ufs datapath: ./ filename: {{ atmosphere_ensemble_analysis_prefix }}cubed_sphere_grid_catmi{{ '%03d' % atmosphere_iau_hours[ihour] }}.nc + max allowable geometry difference: 1e-4 increment: added variables: - eastward_wind diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 index fbb9937ca..ce77c7ed4 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 @@ -72,6 +72,7 @@ members from template: filenames: - {{ atmosphere_ensemble_history_prefix }}cubed_sphere_grid_atmf006.nc - {{ atmosphere_ensemble_history_prefix }}cubed_sphere_grid_sfcf006.nc + max allowable geometry difference: 1e-4 datetime: '{{ atmosphere_background_time_iso }}' state variables: *bkgvars field io names: &field_io_names diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 index b23d2f65f..8280f54e7 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 @@ -72,6 +72,7 @@ members: filenames: - {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_atmf006.nc - {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_sfcf006.nc + max allowable geometry difference: 1e-4 state variables: *bkgvars field io names: &field_io_names eastward_wind: ugrd diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 index 86ea4014e..f5a10238a 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background.yaml.j2 @@ -5,6 +5,7 @@ datetime: "{{ atmosphere_background_time_iso }}" filenames: - {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_atmf006.nc - {{ atmosphere_variational_history_prefix }}cubed_sphere_grid_sfcf006.nc +max allowable geometry difference: 1e-4 state variables: - eastward_wind - northward_wind diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_background_ensemble.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background_ensemble.yaml.j2 index 7ee0f97c0..a9e96487c 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_background_ensemble.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background_ensemble.yaml.j2 @@ -65,6 +65,7 @@ members from template: filenames: - {{ atmosphere_ensemble_history_prefix }}cubed_sphere_grid_atmf006.nc - {{ atmosphere_ensemble_history_prefix }}cubed_sphere_grid_sfcf006.nc + max allowable geometry difference: 1e-4 pattern: "{{ atmosphere_ensemble_pattern | default("%mem%", true) }}" nmembers: {{ atmosphere_number_ensemble_members }} zero padding: 3 diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 index 9f6a59f36..80b3f489e 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 @@ -82,6 +82,7 @@ components: filenames: - {{ atmosphere_ensemble_history_prefix }}cubed_sphere_grid_atmf006.nc - {{ atmosphere_ensemble_history_prefix }}cubed_sphere_grid_sfcf006.nc + max allowable geometry difference: 1e-4 pattern: "{{ atmosphere_ensemble_pattern | default("%mem%", true) }}" nmembers: {{atmosphere_number_ensemble_members}} zero padding: 3 From 65b0b4b775fe5a1039c9533124d6d91b3b4c941c Mon Sep 17 00:00:00 2001 From: DavidNew-NOAA <134300700+DavidNew-NOAA@users.noreply.github.com> Date: Fri, 13 Jun 2025 14:46:30 -0400 Subject: [PATCH 130/199] Save soca_diag_stats yaml with JCB (#141) This PR moves the saving of the soca_diag_stats yaml into the JCB template that already saves the obs statistics. Issue: https://github.com/NOAA-EMC/GDASApp/issues/1738 Companions PRs: https://github.com/NOAA-EMC/GDASApp/pull/1748 https://github.com/NOAA-EMC/global-workflow/pull/3797 --- parm/jcb-gdas/algorithm/marine/soca_diags_finalize.yaml.j2 | 1 + 1 file changed, 1 insertion(+) diff --git a/parm/jcb-gdas/algorithm/marine/soca_diags_finalize.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_diags_finalize.yaml.j2 index 3f1f1e3f6..a05143b34 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_diags_finalize.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_diags_finalize.yaml.j2 @@ -5,3 +5,4 @@ copy_opt: - ['{{marine_obsdataout_path}}/{{marine_obsdatain_prefix}}ocn.{{observation_from_jcb}}.stats.csv', '{{marine_obsdatastats_path}}'] {% endif %} {% endfor %} +- ['{{marine_output_dir}}/soca_diag_stats.yaml', '{{marine_config_path}}'] From 91e7473aa12cbbf585a21ee7a309012080d17073 Mon Sep 17 00:00:00 2001 From: JohnSteffen-NOAA <77806417+JohnSteffen-NOAA@users.noreply.github.com> Date: Wed, 18 Jun 2025 10:18:44 -0400 Subject: [PATCH 131/199] Marine configurations for cp4.02c-parallel experiments (#140) The largest changes include modifying the error coefficients for ADT, setting the AMSR2 icec retrievals to monitoring mode and other icec retrievals to be assimilated, the tropical moorings are now monitored and a new obs space for the surface drifters is included. --------- Co-authored-by: john steffen Co-authored-by: john steffen Co-authored-by: Guillaume Vernieres --- .../observations/marine/adt_rads_all.yaml.j2 | 2 +- .../marine/icec_abi_g16_l2.yaml.j2 | 14 +- .../marine/icec_amsu_mb_l2.yaml.j2 | 14 +- .../marine/icec_atms_n20_l2.yaml.j2 | 14 +- .../marine/icec_atms_n21_l2.yaml.j2 | 14 +- .../marine/icec_atms_npp_l2.yaml.j2 | 14 +- .../marine/icec_gmi_gpm_l2.yaml.j2 | 14 +- .../marine/icec_ssmis_f17_l2.yaml.j2 | 14 +- .../marine/icec_viirs_n20_l2_north.yaml.j2 | 2 +- .../marine/icec_viirs_n20_l2_south.yaml.j2 | 2 +- .../marine/insitu_profile_argo.yaml.j2 | 348 ------------------ .../marine/insitu_profile_dbuoy.yaml.j2 | 43 --- .../marine/insitu_profile_dbuoyb.yaml.j2 | 42 --- .../marine/insitu_profile_mbuoy.yaml.j2 | 42 --- .../marine/insitu_profile_mbuoyb.yaml.j2 | 42 --- .../marine/insitu_profile_tropical.yaml.j2 | 8 +- .../insitu_surface_dbuoyb_drifter.yaml.j2 | 56 +++ .../marine/insitu_surface_drifter.yaml.j2 | 42 --- 18 files changed, 115 insertions(+), 612 deletions(-) delete mode 100644 parm/jcb-gdas/observations/marine/insitu_profile_argo.yaml.j2 delete mode 100644 parm/jcb-gdas/observations/marine/insitu_profile_dbuoy.yaml.j2 delete mode 100644 parm/jcb-gdas/observations/marine/insitu_profile_dbuoyb.yaml.j2 delete mode 100644 parm/jcb-gdas/observations/marine/insitu_profile_mbuoy.yaml.j2 delete mode 100644 parm/jcb-gdas/observations/marine/insitu_profile_mbuoyb.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/insitu_surface_dbuoyb_drifter.yaml.j2 delete mode 100644 parm/jcb-gdas/observations/marine/insitu_surface_drifter.yaml.j2 diff --git a/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 b/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 index 9c75aa5e6..d9441cf37 100644 --- a/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 @@ -50,7 +50,7 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 100e3 + minvalue: 50e3 - filter: BlackList where: - variable: diff --git a/parm/jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 index 95c3fe345..7c6cbdc66 100644 --- a/parm/jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 @@ -43,19 +43,19 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 100e3 + minvalue: 0e3 - filter: Gaussian Thinning horizontal_mesh: 25 #km use_reduced_horizontal_grid: true select_median: true action: name: reduce obs space - - filter: Domain Check - action: - name: passivate - where: - - variable: {name: GeoVaLs/sea_surface_temperature} - maxvalue: -4.0 +# - filter: Domain Check +# action: +# name: passivate +# where: +# - variable: {name: GeoVaLs/sea_surface_temperature} +# maxvalue: -4.0 {% if letkf_app | default(false) %} obs localizations: - localization method: Rossby diff --git a/parm/jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 index 95c3fe345..7c6cbdc66 100644 --- a/parm/jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 @@ -43,19 +43,19 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 100e3 + minvalue: 0e3 - filter: Gaussian Thinning horizontal_mesh: 25 #km use_reduced_horizontal_grid: true select_median: true action: name: reduce obs space - - filter: Domain Check - action: - name: passivate - where: - - variable: {name: GeoVaLs/sea_surface_temperature} - maxvalue: -4.0 +# - filter: Domain Check +# action: +# name: passivate +# where: +# - variable: {name: GeoVaLs/sea_surface_temperature} +# maxvalue: -4.0 {% if letkf_app | default(false) %} obs localizations: - localization method: Rossby diff --git a/parm/jcb-gdas/observations/marine/icec_atms_n20_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_atms_n20_l2.yaml.j2 index 95c3fe345..7c6cbdc66 100644 --- a/parm/jcb-gdas/observations/marine/icec_atms_n20_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_atms_n20_l2.yaml.j2 @@ -43,19 +43,19 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 100e3 + minvalue: 0e3 - filter: Gaussian Thinning horizontal_mesh: 25 #km use_reduced_horizontal_grid: true select_median: true action: name: reduce obs space - - filter: Domain Check - action: - name: passivate - where: - - variable: {name: GeoVaLs/sea_surface_temperature} - maxvalue: -4.0 +# - filter: Domain Check +# action: +# name: passivate +# where: +# - variable: {name: GeoVaLs/sea_surface_temperature} +# maxvalue: -4.0 {% if letkf_app | default(false) %} obs localizations: - localization method: Rossby diff --git a/parm/jcb-gdas/observations/marine/icec_atms_n21_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_atms_n21_l2.yaml.j2 index 95c3fe345..7c6cbdc66 100644 --- a/parm/jcb-gdas/observations/marine/icec_atms_n21_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_atms_n21_l2.yaml.j2 @@ -43,19 +43,19 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 100e3 + minvalue: 0e3 - filter: Gaussian Thinning horizontal_mesh: 25 #km use_reduced_horizontal_grid: true select_median: true action: name: reduce obs space - - filter: Domain Check - action: - name: passivate - where: - - variable: {name: GeoVaLs/sea_surface_temperature} - maxvalue: -4.0 +# - filter: Domain Check +# action: +# name: passivate +# where: +# - variable: {name: GeoVaLs/sea_surface_temperature} +# maxvalue: -4.0 {% if letkf_app | default(false) %} obs localizations: - localization method: Rossby diff --git a/parm/jcb-gdas/observations/marine/icec_atms_npp_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_atms_npp_l2.yaml.j2 index 95c3fe345..7c6cbdc66 100644 --- a/parm/jcb-gdas/observations/marine/icec_atms_npp_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_atms_npp_l2.yaml.j2 @@ -43,19 +43,19 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 100e3 + minvalue: 0e3 - filter: Gaussian Thinning horizontal_mesh: 25 #km use_reduced_horizontal_grid: true select_median: true action: name: reduce obs space - - filter: Domain Check - action: - name: passivate - where: - - variable: {name: GeoVaLs/sea_surface_temperature} - maxvalue: -4.0 +# - filter: Domain Check +# action: +# name: passivate +# where: +# - variable: {name: GeoVaLs/sea_surface_temperature} +# maxvalue: -4.0 {% if letkf_app | default(false) %} obs localizations: - localization method: Rossby diff --git a/parm/jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 index ed60b17d3..695b22409 100644 --- a/parm/jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 @@ -43,13 +43,13 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 100e3 - - filter: Domain Check - action: - name: passivate - where: - - variable: {name: GeoVaLs/sea_surface_temperature} - maxvalue: -4.0 + minvalue: 0e3 +# - filter: Domain Check +# action: +# name: passivate +# where: +# - variable: {name: GeoVaLs/sea_surface_temperature} +# maxvalue: -4.0 {% if letkf_app | default(false) %} obs localizations: - localization method: Rossby diff --git a/parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 index 95c3fe345..7c6cbdc66 100644 --- a/parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 @@ -43,19 +43,19 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 100e3 + minvalue: 0e3 - filter: Gaussian Thinning horizontal_mesh: 25 #km use_reduced_horizontal_grid: true select_median: true action: name: reduce obs space - - filter: Domain Check - action: - name: passivate - where: - - variable: {name: GeoVaLs/sea_surface_temperature} - maxvalue: -4.0 +# - filter: Domain Check +# action: +# name: passivate +# where: +# - variable: {name: GeoVaLs/sea_surface_temperature} +# maxvalue: -4.0 {% if letkf_app | default(false) %} obs localizations: - localization method: Rossby diff --git a/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_north.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_north.yaml.j2 index 95c3fe345..54ae75f2b 100644 --- a/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_north.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_north.yaml.j2 @@ -43,7 +43,7 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 100e3 + minvalue: 0e3 - filter: Gaussian Thinning horizontal_mesh: 25 #km use_reduced_horizontal_grid: true diff --git a/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_south.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_south.yaml.j2 index 95c3fe345..54ae75f2b 100644 --- a/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_south.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_south.yaml.j2 @@ -43,7 +43,7 @@ - filter: Domain Check where: - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 100e3 + minvalue: 0e3 - filter: Gaussian Thinning horizontal_mesh: 25 #km use_reduced_horizontal_grid: true diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_argo.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_argo.yaml.j2 deleted file mode 100644 index 2b08a67bc..000000000 --- a/parm/jcb-gdas/observations/marine/insitu_profile_argo.yaml.j2 +++ /dev/null @@ -1,348 +0,0 @@ -- obs space: - name: {{observation_from_jcb}} - obsdatain: - engine: - type: H5File - obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" - obsgrouping: - group variables: [latitude, longitude, dateTime] - sort variable: depth - sort group: MetaData - sort order: ascending - obsdataout: - engine: - type: H5File - obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" - simulated variables: [waterTemperature, salinity] - observed variables: [waterTemperature, salinity] - io pool: - max pool size: 1 - obs operator: - name: Composite - components: - - name: InsituTemperature - variables: - - name: waterTemperature - - name: VertInterp - # note: alias file is needed for now because of a conflict with "waterTemperature". - # The built-in alias maps to "ocean_temperature", but soca uses "sea_water_temperature" - observation alias file: obsop_name_map.yaml - variables: - - name: salinity - vertical coordinate: sea_water_depth - observation vertical coordinate: depth - interpolation method: linear - obs error: - covariance model: diagonal - - #------------------------------------------------------------------------------- - # START OF OBS FILTERS (work done by Kriti Bhargava) - # The QC filters used here are based on the document by IODE that can be found at - # https://cdn.ioos.noaa.gov/media/2017/12/recommendations_in_situ_data_real_time_qc.pdf - #------------------------------------------------------------------------------- - obs filters: - - # land check - - filter: Domain Check - where: - - variable: {name: GeoVaLs/sea_area_fraction} - value: is_valid - minvalue: 0.5 - - #------------------------------------------------------------------------------- - ## Filters for T: - #------------------------------------------------------------------------------- - #------------------------------------------------------------------------------- - ### Global range test - #----------------------------------------------------------------------------- - - filter: Bounds Check - filter variables: [{name: waterTemperature}] - minvalue: -2.5 - maxvalue: 40.0 - - #----------------------------------------------------------------------------- - ### Regional range tests - #----------------------------------------------------------------------------- - - #### Red Sea - #----------------------------------------------------------------------------- - #### - #### the document linked here describes Red sea as the are between 10N, 40E; - #### 20N, 50E; 30N, 30E; 10N, 40E. But that would also include Gulf of Aden. - #### A more reasonable choice seemed to be a box that extends from 10 N to - #### 30 N and 30 E to 45 East . - - - filter: Bounds Check - filter variables: [{name: waterTemperature}] - minvalue: 21.7 - maxvalue: 40.0 - where: - - variable: - name: MetaData/latitude - minvalue: 10 - maxvalue: 30 - - variable: - name: MetaData/longitude - minvalue: 30 - maxvalue: 45 - - #### Mediterranean Sea - #----------------------------------------------------------------------------- - ##### Area 1/3 for Mediterranean Sea - - filter: Bounds Check - filter variables: [{name: waterTemperature}] - minvalue: 10.0 - maxvalue: 40.0 - where: - - variable: - name: MetaData/latitude - minvalue: 30 - maxvalue: 40 - - variable: - name: MetaData/longitude - minvalue: -6 - maxvalue: 40 - ##### Area 2/3 for Mediterranean Sea - - filter: Bounds Check - filter variables: [{name: waterTemperature}] - minvalue: 10.0 - maxvalue: 40.0 - where: - - variable: - name: MetaData/latitude - minvalue: 40 - maxvalue: 41.5 - - variable: - name: MetaData/longitude - minvalue: 20 - maxvalue: 30 - ##### Area 3/3 for Mediterranean Sea - - filter: Bounds Check - filter variables: [{name: waterTemperature}] - minvalue: 10.0 - maxvalue: 40.0 - where: - - variable: - name: MetaData/latitude - minvalue: 40 - maxvalue: 46 - - variable: - name: MetaData/longitude - minvalue: 0 - maxvalue: 20 - - #### Northwestern shelves - #----------------------------------------------------------------------------- - - filter: Bounds Check - filter variables: [{name: waterTemperature}] - minvalue: -2.0 - maxvalue: 24.0 - where: - - variable: - name: MetaData/latitude - minvalue: 50 - maxvalue: 60 - - variable: - name: MetaData/longitude - minvalue: -20 - maxvalue: 10 - #### Southwestern shelves - #----------------------------------------------------------------------------- - - filter: Bounds Check - filter variables: [{name: waterTemperature}] - minvalue: -2.0 - maxvalue: 30 - where: - - variable: - name: MetaData/latitude - minvalue: 25 - maxvalue: 50 - - variable: - name: MetaData/longitude - minvalue: -30 - maxvalue: 0 - - #### Arctic Ocean - #----------------------------------------------------------------------------- - - filter: Bounds Check - filter variables: [{name: waterTemperature}] - minvalue: -1.92 - maxvalue: 25.0 - where: - - variable: - name: MetaData/latitude - minvalue: 60 - - - filter: Background Check - filter variables: [{name: waterTemperature}] - threshold: 5.0 - absolute threshold: 5.0 - - ## Filters for S: - #------------------------------------------------------------------------------- - #----------------------------------------------------------------------------- - ### Global range test - #----------------------------------------------------------------------------- - - filter: Bounds Check - filter variables: [{name: salinity}] - minvalue: 2.0 - maxvalue: 41.0 - - #----------------------------------------------------------------------------- - ### Regional range test - #----------------------------------------------------------------------------- - #### Red Sea - #----------------------------------------------------------------------------- - #### - #### the document linked here describes Red sea as the are between 10N, 40E; - #### 20N, 50E; 30N, 30E; 10N, 40E. But that would also include Gulf of Aden. - #### A more reasonable choice seemed to be a box that extends from 10 N to - #### 30 N and 30 E to 45 East . - - - filter: Bounds Check - filter variables: [{name: salinity}] - minvalue: 2.0 - maxvalue: 41.0 - where: - - variable: - name: MetaData/latitude - minvalue: 10 - maxvalue: 30 - - variable: - name: MetaData/longitude - minvalue: 30 - maxvalue: 45 - - #### Mediterranean Sea - #----------------------------------------------------------------------------- - ##### Area 1/3 for Mediterranean Sea - - filter: Bounds Check - filter variables: [{name: salinity}] - minvalue: 2.0 - maxvalue: 40.0 - where: - - variable: - name: MetaData/latitude - minvalue: 30 - maxvalue: 40 - - variable: - name: MetaData/longitude - minvalue: -6 - maxvalue: 40 - ##### Area 2/3 for Mediterranean Sea - - filter: Bounds Check - filter variables: [{name: salinity}] - minvalue: 2.0 - maxvalue: 40.0 - where: - - variable: - name: MetaData/latitude - minvalue: 40 - maxvalue: 41.5 - - variable: - name: MetaData/longitude - minvalue: 20 - maxvalue: 30 - ##### Area 3/3 for Mediterranean Sea - - filter: Bounds Check - filter variables: [{name: salinity}] - minvalue: 2.0 - maxvalue: 40.0 - where: - - variable: - name: MetaData/latitude - minvalue: 40 - maxvalue: 46 - - variable: - name: MetaData/longitude - minvalue: 0 - maxvalue: 20 - - - #### Northwestern shelves - #----------------------------------------------------------------------------- - - filter: Bounds Check - filter variables: [{name: salinity}] - minvalue: 0.0 - maxvalue: 37.0 - where: - - variable: - name: MetaData/latitude - minvalue: 50 - maxvalue: 60 - - variable: - name: MetaData/longitude - minvalue: -20 - maxvalue: 10 - - #### Southwestern shelves - #----------------------------------------------------------------------------- - - filter: Bounds Check - filter variables: [{name: salinity}] - minvalue: 0.0 - maxvalue: 38 - where: - - variable: - name: MetaData/latitude - minvalue: 25 - maxvalue: 50 - - variable: - name: MetaData/longitude - minvalue: -30 - maxvalue: 0 - - #### Arctic Ocean - #----------------------------------------------------------------------------- - - filter: Bounds Check - filter variables: [{name: salinity}] - minvalue: 2.0 - maxvalue: 40.0 - where: - - variable: - name: MetaData/latitude - minvalue: 60 - - - filter: Background Check - filter variables: [{name: salinity}] - threshold: 5.0 - absolute threshold: 5.0 - - #----------------------------------------------------------------------------- - - filter: RejectList - where: - - variable: QCflagsData/waterTemperature - minvalue: 1 - defer to post: true - - #-------------------------------------------------------------------------- - ### Set in situ obs multiplier coef for T,S - #------------------------------------------------------------------------ - - filter: Perform Action - action: - name: assign error - error function: - name: ObsFunction/LinearCombination - options: - variables: - - ObsError/waterTemperature - coefs: - - 1000.0 - - filter: Perform Action - action: - name: assign error - error function: - name: ObsFunction/LinearCombination - options: - variables: - - ObsError/salinity - coefs: - - 1000.0 -{% if letkf_app | default(false) %} - obs localizations: - - localization method: Rossby - base value: 100.0e3 - rossby mult: 1.0 - min grid mult: 2.0 - min value: 200.0e3 - max value: 900.0e3 -{% endif %} diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_dbuoy.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_dbuoy.yaml.j2 deleted file mode 100644 index 2848b56aa..000000000 --- a/parm/jcb-gdas/observations/marine/insitu_profile_dbuoy.yaml.j2 +++ /dev/null @@ -1,43 +0,0 @@ -- obs space: - name: {{observation_from_jcb}} - obsdatain: - engine: - type: H5File - obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" - obsdataout: - engine: - type: H5File - obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" - simulated variables: [waterTemperature] - observed variables: [waterTemperature] - io pool: - max pool size: 1 - obs operator: - name: Composite - components: - - name: InsituTemperature - variables: - - name: waterTemperature - obs error: - covariance model: diagonal - obs filters: - - filter: Perform Action - action: - name: assign error - error function: - name: ObsFunction/LinearCombination - options: - variables: - - ObsError/waterTemperature - coefs: - - 1000.0 - -{% if letkf_app | default(false) %} - obs localizations: - - localization method: Rossby - base value: 100.0e3 - rossby mult: 1.0 - min grid mult: 2.0 - min value: 200.0e3 - max value: 900.0e3 -{% endif %} diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_dbuoyb.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_dbuoyb.yaml.j2 deleted file mode 100644 index 65a5a704e..000000000 --- a/parm/jcb-gdas/observations/marine/insitu_profile_dbuoyb.yaml.j2 +++ /dev/null @@ -1,42 +0,0 @@ -- obs space: - name: {{observation_from_jcb}} - obsdatain: - engine: - type: H5File - obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" - obsdataout: - engine: - type: H5File - obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" - simulated variables: [waterTemperature] - observed variables: [waterTemperature] - io pool: - max pool size: 1 - obs operator: - name: Composite - components: - - name: InsituTemperature - variables: - - name: waterTemperature - obs error: - covariance model: diagonal - obs filters: - - filter: Perform Action - action: - name: assign error - error function: - name: ObsFunction/LinearCombination - options: - variables: - - ObsError/waterTemperature - coefs: - - 1000.0 -{% if letkf_app | default(false) %} - obs localizations: - - localization method: Rossby - base value: 100.0e3 - rossby mult: 1.0 - min grid mult: 2.0 - min value: 200.0e3 - max value: 900.0e3 -{% endif %} diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_mbuoy.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_mbuoy.yaml.j2 deleted file mode 100644 index 65a5a704e..000000000 --- a/parm/jcb-gdas/observations/marine/insitu_profile_mbuoy.yaml.j2 +++ /dev/null @@ -1,42 +0,0 @@ -- obs space: - name: {{observation_from_jcb}} - obsdatain: - engine: - type: H5File - obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" - obsdataout: - engine: - type: H5File - obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" - simulated variables: [waterTemperature] - observed variables: [waterTemperature] - io pool: - max pool size: 1 - obs operator: - name: Composite - components: - - name: InsituTemperature - variables: - - name: waterTemperature - obs error: - covariance model: diagonal - obs filters: - - filter: Perform Action - action: - name: assign error - error function: - name: ObsFunction/LinearCombination - options: - variables: - - ObsError/waterTemperature - coefs: - - 1000.0 -{% if letkf_app | default(false) %} - obs localizations: - - localization method: Rossby - base value: 100.0e3 - rossby mult: 1.0 - min grid mult: 2.0 - min value: 200.0e3 - max value: 900.0e3 -{% endif %} diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_mbuoyb.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_mbuoyb.yaml.j2 deleted file mode 100644 index 65a5a704e..000000000 --- a/parm/jcb-gdas/observations/marine/insitu_profile_mbuoyb.yaml.j2 +++ /dev/null @@ -1,42 +0,0 @@ -- obs space: - name: {{observation_from_jcb}} - obsdatain: - engine: - type: H5File - obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" - obsdataout: - engine: - type: H5File - obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" - simulated variables: [waterTemperature] - observed variables: [waterTemperature] - io pool: - max pool size: 1 - obs operator: - name: Composite - components: - - name: InsituTemperature - variables: - - name: waterTemperature - obs error: - covariance model: diagonal - obs filters: - - filter: Perform Action - action: - name: assign error - error function: - name: ObsFunction/LinearCombination - options: - variables: - - ObsError/waterTemperature - coefs: - - 1000.0 -{% if letkf_app | default(false) %} - obs localizations: - - localization method: Rossby - base value: 100.0e3 - rossby mult: 1.0 - min grid mult: 2.0 - min value: 200.0e3 - max value: 900.0e3 -{% endif %} diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_tropical.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_tropical.yaml.j2 index 65a5a704e..238b81762 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_tropical.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_tropical.yaml.j2 @@ -30,7 +30,13 @@ variables: - ObsError/waterTemperature coefs: - - 1000.0 + - 100.0 + - filter: Domain Check + action: + name: passivate + where: + - variable: {name: GeoVaLs/sea_surface_temperature} + maxvalue: -4.0 {% if letkf_app | default(false) %} obs localizations: - localization method: Rossby diff --git a/parm/jcb-gdas/observations/marine/insitu_surface_dbuoyb_drifter.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_surface_dbuoyb_drifter.yaml.j2 new file mode 100644 index 000000000..d3a1a4a3f --- /dev/null +++ b/parm/jcb-gdas/observations/marine/insitu_surface_dbuoyb_drifter.yaml.j2 @@ -0,0 +1,56 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaSurfaceTemperature] + observed variables: [seaSurfaceTemperature] + io pool: + max pool size: 1 + get values: + time interpolation: linear + obs operator: + name: Identity + observation alias file: obsop_name_map.yaml + obs error: + covariance model: diagonal + + obs pre filters: + - filter: Bounds Check + minvalue: -1.2 + maxvalue: 41.0 + - filter: Domain Check + where: + - variable: {name: ObsError/seaSurfaceTemperature} + minvalue: 1.0e-8 + + obs prior filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + value: is_valid + minvalue: 0.9 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_ice_area_fraction} + maxvalue: 1.0e-5 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_surface_temperature} + minvalue: -1.2 + +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} + diff --git a/parm/jcb-gdas/observations/marine/insitu_surface_drifter.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_surface_drifter.yaml.j2 deleted file mode 100644 index 65a5a704e..000000000 --- a/parm/jcb-gdas/observations/marine/insitu_surface_drifter.yaml.j2 +++ /dev/null @@ -1,42 +0,0 @@ -- obs space: - name: {{observation_from_jcb}} - obsdatain: - engine: - type: H5File - obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" - obsdataout: - engine: - type: H5File - obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" - simulated variables: [waterTemperature] - observed variables: [waterTemperature] - io pool: - max pool size: 1 - obs operator: - name: Composite - components: - - name: InsituTemperature - variables: - - name: waterTemperature - obs error: - covariance model: diagonal - obs filters: - - filter: Perform Action - action: - name: assign error - error function: - name: ObsFunction/LinearCombination - options: - variables: - - ObsError/waterTemperature - coefs: - - 1000.0 -{% if letkf_app | default(false) %} - obs localizations: - - localization method: Rossby - base value: 100.0e3 - rossby mult: 1.0 - min grid mult: 2.0 - min value: 200.0e3 - max value: 900.0e3 -{% endif %} From 8a80f7e72ec5f97748b8b9ed03250a5fd4b5d887 Mon Sep 17 00:00:00 2001 From: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com> Date: Wed, 18 Jun 2025 21:17:09 -0400 Subject: [PATCH 132/199] add Deterministic to local ensemble DA solver name (#143) --- .../test/client_integration/gdas-atmosphere-templates.yaml | 2 +- .../jcb-gdas/test/client_integration/gdas-marine-templates.yaml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml b/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml index 2b64a1966..8425b80f6 100644 --- a/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml +++ b/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml @@ -118,7 +118,7 @@ atmosphere_obsbiascovout_suffix: ".satbias_cov.nc" # Local Ensemble DA (LETKF) # ------------------------- -local_ensemble_da_solver: GETKF +local_ensemble_da_solver: Deterministic GETKF increment_variables: [ua,va,DZ,delp,t,ps,sphum,ice_wat,liq_wat,o3mr] diff --git a/parm/jcb-gdas/test/client_integration/gdas-marine-templates.yaml b/parm/jcb-gdas/test/client_integration/gdas-marine-templates.yaml index 238b4a83d..b99a221bc 100644 --- a/parm/jcb-gdas/test/client_integration/gdas-marine-templates.yaml +++ b/parm/jcb-gdas/test/client_integration/gdas-marine-templates.yaml @@ -87,7 +87,7 @@ marine_obsdataout_suffix: "_2024020100.nc" # Local Ensemble DA (LETKF) # ------------------------- -local_ensemble_da_solver: GETKF +local_ensemble_da_solver: Deterministic GETKF increment_variables: [cicen, hicen, hsnon, socn, tocn, uocn, vocn, ssh] From 50ef31ce03dc84b9b745a68fccb1f34ad35b45b8 Mon Sep 17 00:00:00 2001 From: NicholasEsposito-NOAA <62616739+nicholasesposito@users.noreply.github.com> Date: Wed, 25 Jun 2025 15:19:06 -0400 Subject: [PATCH 133/199] Add aircraft yaml j2. (#103) This PR adds the aircraft profiles prepbufr yaml (phase one) to the repository. This solves issue [JEDI T20 #116](https://github.com/issues/assigned?issue=NOAA-EMC%7CJEDI-T2O%7C116) This should be merged at the same time as GDASApp [PR#1597](https://github.com/NOAA-EMC/GDASApp/pull/1597/files) Many of the paths are the ones I used to test the yaml. I added what I believe to be the jcb paths though they will likely need work. --------- Co-authored-by: Nicholas Esposito Co-authored-by: Nicholas Esposito Co-authored-by: Nicholas Esposito --- .../observations/atmosphere/aircraft.yaml.j2 | 356 ------------------ .../atmosphere/aircraft_humidity.yaml.j2 | 180 +++++++++ .../atmosphere/aircraft_temperature.yaml.j2 | 298 +++++++++++++++ .../atmosphere/aircraft_wind.yaml.j2 | 289 ++++++++++++++ .../gdas-atmosphere-templates.yaml | 4 + 5 files changed, 771 insertions(+), 356 deletions(-) delete mode 100644 parm/jcb-gdas/observations/atmosphere/aircraft.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/aircraft_humidity.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/aircraft_temperature.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/aircraft_wind.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/aircraft.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/aircraft.yaml.j2 deleted file mode 100644 index f5e081719..000000000 --- a/parm/jcb-gdas/observations/atmosphere/aircraft.yaml.j2 +++ /dev/null @@ -1,356 +0,0 @@ -- - - # Observation Space (I/O) - # ----------------------- - obs space: - name: Aircraft - obsdatain: - engine: - type: H5File - obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" - obsgrouping: - group variables: ["stationIdentification"] - sort variable: "pressure" - sort order: "descending" - obsdataout: - engine: - type: H5File - obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" - io pool: - max pool size: 1 - simulated variables: [windEastward, windNorthward, airTemperature, specificHumidity] - # - - # Observation Operator - # -------------------- - obs operator: - name: VertInterp - # - - # Observation Filters (QC) - # ------------------------ - obs filters: - # Observation Range Sanity Check - - filter: Bounds Check - filter variables: - - name: airTemperature - minvalue: 195 - maxvalue: 327 - action: - name: reject - # - - filter: Bounds Check - filter variables: - - name: specificHumidity - minvalue: 1.0E-7 - maxvalue: 0.034999999 - action: - name: reject - # - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - minvalue: -130 - maxvalue: 130 - action: - name: reject - # - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - test variables: - - name: ObsFunction/Velocity - maxvalue: 130.0 - action: - name: reject - # Reject when pressure is less than 126 mb. - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - test variables: - - name: MetaData/pressure - minvalue: 12600 - action: - name: reject - # - # Reject all obs with PreQC mark already set above 3 - # - filter: PreQC - # maxvalue: 3 - # action: - # name: reject - # - #-------------------------------------------------------------------------------------------------------------------- - # Wind - #-------------------------------------------------------------------------------------------------------------------- - # - # Begin by assigning all ObsError to a constant value. These will get overwritten (as needed) for specific types. - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - action: - name: assign error - error parameter: 2.0 # 2.0 m/s - # Assign intial ObsError specific to AIREP/ACARS - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - action: - name: assign error - error parameter: 3.6 # 3.6 m/s - where: - - variable: - name: ObsType/windEastward - is_in: 230 - # Assign intial ObsError specific to AMDAR - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - action: - name: assign error - error parameter: 3.0 # 3.0 m/s - where: - - variable: - name: ObsType/windEastward - is_in: 231 - # Assign intial ObsError specific to MDCRS - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - action: - name: assign error - error parameter: 2.5 # 2.5 m/s - where: - - variable: - name: ObsType/windEastward - is_in: 233 - # - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [100000, 95000, 80000, 65000, 60000, 55000, 50000, 45000, 40000, - 35000, 30000, 25000, 20000, 15000, 10000] #Pressure (Pa) - errors: [1.4, 1.5, 1.6, 1.8, 1.9, 2.0, 2.1, 2.3, 2.6, 2.8, 3.0, 3.2, 2.7, - 2.4, 2.1] - # Assign the initial ObsError, based on height/pressure for RECON aircraft - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [70000, 65000, 60000, 55000, 50000, 45000, 40000, 35000, 30000, 25000, - 20000, 15000, 10000, 7500, 5000] - errors: [2.4, 2.5, 2.6, 2.7, 2.8, 2.95, 3.1, 3.25, 3.4, 3.175, 2.95, 2.725, - 2.5, 2.6, 2.7] - where: - - variable: - name: ObsType/windEastward - is_in: 232 - # Reject when difference of wind direction is more than 50 degrees. - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - test variables: - - name: ObsFunction/WindDirAngleDiff - options: - minimum_uv: 3.5 - maxvalue: 50.0 - action: - name: reject - defer to post: true - # When multiple obs exist within a single vertical model level, inflate ObsError - # - filter: Perform Action - # filter variables: - # - name: windEastward - # action: - # name: inflate error - # inflation variable: - # name: ObsFunction/ObsErrorFactorConventional - # options: - # test QCflag: PreQC - # inflate variables: [windEastward] - # defer to post: true - # - # - filter: Perform Action - # filter variables: - # - name: windNorthward - # action: - # name: inflate error - # inflation variable: - # name: ObsFunction/ObsErrorFactorConventional - # options: - # test QCflag: PreQC - # inflate variables: [windNorthward] - # defer to post: true - # - # Gross error check with (O - B) / ObsError greater than threshold. - - filter: Background Check - filter variables: - - name: windEastward - - name: windNorthward - threshold: 6.0 - absolute threshold: 19.0 - action: - name: reject - # - #-------------------------------------------------------------------------------------------------------------------- - # Temperature - #-------------------------------------------------------------------------------------------------------------------- - # - # Begin by assigning all ObsError to a constant value. These will get overwritten for specific types. - - filter: Perform Action - filter variables: - - name: airTemperature - action: - name: assign error - error parameter: 2.0 # 2.0 K - # Assign the initial observation error, based on pressure (for AIREP/ACARS; itype=130) - - filter: Perform Action - filter variables: - - name: airTemperature - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [100000, 95000, 90000, 85000, 80000] - errors: [2.5, 2.3, 2.1, 1.9, 1.7] - where: - - variable: - name: ObsType/airTemperature - is_in: 130 - # Assign the initial observation error, based on pressure (for AMDAR and MDCRS; itype=131,133) - - filter: Perform Action - filter variables: - - name: airTemperature - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [100000, 95000, 90000, 85000, 80000] - errors: [1.4706, 1.3529, 1.2353, 1.1176, 1.0] - where: - - variable: - name: ObsType/airTemperature - is_in: 131,133 - # Assign the initial observation error, based on pressure (for RECON aircraft; itype=132) - - filter: Perform Action - filter variables: - - name: airTemperature - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [100000, 95000, 90000, 85000, 35000, 30000, 25000, 20000, 15000, - 10000, 7500, 5000, 4000, 3200, 2000, 1000] - errors: [1.2, 1.1, 0.9, 0.8, 0.8, 0.9, 1.2, 1.2, 1.0, 0.8, 0.8, 0.9, 0.95, - 1.0, 1.25, 1.5] - where: - - variable: - name: ObsType/airTemperature - is_in: 132 - # When multiple obs exist within a single vertical model level, inflate ObsError - # - filter: Perform Action - # filter variables: - # - name: airTemperature - # action: - # name: inflate error - # inflation variable: - # name: ObsFunction/ObsErrorFactorConventional - # options: - # test QCflag: PreQC - # inflate variables: [airTemperature] - # defer to post: true - # Gross error check with (O - B) / ObsError greater than threshold. - - filter: Background Check - filter variables: - - name: airTemperature - threshold: 7.0 - absolute threshold: 8.0 - action: - name: reject - # - #-------------------------------------------------------------------------------------------------------------------- - # Moisture - #-------------------------------------------------------------------------------------------------------------------- - # - # Assign the initial observation error, based on height/pressure ONLY MDCRS - - filter: Perform Action - filter variables: - - name: specificHumidity - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [110000, 105000, 100000, 95000, 90000, 85000, 80000, 75000, 70000, - 65000, 60000, 55000, 50000, 45000, 40000, 35000, 30000, 25000, 20000, - 15000, 10000, 7500, 5000, 4000, 3000] - errors: [.19455, .19062, .18488, .17877, .17342, .16976, .16777, .16696, - .16605, .16522, .16637, .17086, .17791, .18492, .18996, .19294, .19447, - .19597, .19748, .19866, .19941, .19979, .19994, .19999, .2] - scale_factor_var: ObsValue/specificHumidity - where: - - variable: - name: ObsType/specificHumidity - is_in: 133 - # When multiple obs exist within a single vertical model level, inflate ObsError - # - filter: Perform Action - # filter variables: - # - name: specificHumidity - # action: - # name: inflate error - # inflation variable: - # name: ObsFunction/ObsErrorFactorConventional - # options: - # test QCflag: PreQC - # inflate variables: [specificHumidity] - # defer to post: true - # Gross error check with (O - B) / ObsError greater than threshold. - - filter: Background Check - filter variables: - - name: specificHumidity - threshold: 8.0 - action: - name: reject - - - # GeoVaLs for Driving Observation Operators (testing mode) - # -------------------------------------------------------- - geovals: - filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" - - # Passed benchmark for UFO testing - # -------------------------------- - passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/aircraft_humidity.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/aircraft_humidity.yaml.j2 new file mode 100644 index 000000000..8bc9cd746 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/aircraft_humidity.yaml.j2 @@ -0,0 +1,180 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: Aircraft + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsgrouping: + group variables: [stationIdentification] + sort variable: pressure + sort order: descending + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [specificHumidity] + + # Observation Operator + # -------------------- + obs operator: + name: Composite + components: + - name: VertInterp + variables: [specificHumidity ] + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: Composite + components: + - name: VertInterp + variables: [specificHumidity] + + obs pre filters: + + # Assign the initial observation error, based on pressure (for AIREP/ACARS; itype=130) + # (for AMDAR itype = 131, and for Canadian AMDAR itype=135) + - filter: Perform Action + filter variables: + - name: specificHumidity + where: + - variable: + name: ObsType/specificHumidity + is_in: 130, 131, 135 + action: + name: reject + #defer to post: true + + # Assign the initial observation error, based on pressure (for RECON aircraft; itype=132) + - filter: Perform Action + filter variables: + - name: specificHumidity + action: + name: assign error + error parameter: 0.2 + where: + - variable: + name: ObsType/specificHumidity + is_in: 132 + + + # Assign the initial observation error, based on pressure (for MDCRS; itype=133) + - filter: Perform Action + filter variables: + - name: specificHumidity + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000, 105000, 100000, 95000, 90000, 85000, 80000, 75000, 70000, 65000, 60000, 55000, 50000, 45000, 40000, 35000, 30000, 25000, 20000, 15000, 10000, 7500, 5000, 4000, 3000] + errors: [.19455, .19062, .18488, .17877, .17342, .16976, .16777, .16696, .16605, .16522, .16637, .17086, .17791, .18492, .18996, .19294, .19447, .19597, .19748, .19866, .19941, .19979, .19994, .19999, .20] + where: + - variable: + name: ObsType/specificHumidity + is_in: 133 + + # Assign the initial observation error, based on pressure (for TAMDAR itype=134) + - filter: Perform Action + filter variables: + - name: specificHumidity + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000, 75000] + errors: [0.2, 1000000000.0 ] + where: + - variable: + name: ObsType/specificHumidity + is_in: 134 + + # Inflate if QC == 3 or 7 + - filter: Perform Action + filter variables: + - name: specificHumidity + action: + name: inflate error + inflation factor: 1.2 + where: + - variable: PreQC/specificHumidity + is_in: 3,7 + + obs prior filters: + + # Calculates ratio_errors from GSI + # Error inflation based on pressure check + - filter: Perform Action + filter variables: + - name: specificHumidity + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: specificHumidity + inflation factor: 8.0 + request_saturation_specific_humidity_geovals: true + geovar_sfc_geomz: height_above_mean_sea_level_at_surface + + + obs post filters: + + # Background check + # ratio threshold 8.0 + - filter: Background Check + test_hofx: HofX + filter variables: + - name: specificHumidity + threshold: 8.0 + action: + name: reject + where: + - variable: PreQC/specificHumidity + is_not_in: 3,7 + defer to post: true + + # ratio threshold * 0.7 if PreQC=3 + - filter: Background Check + test_hofx: HofX + filter variables: + - name: specificHumidity + threshold: 5.6 + action: + name: reject + where: + - variable: PreQC/specificHumidity + is_in: 3,7 + defer to post: true + + + # Reject where QC > 8 + - filter: Perform Action + filter variables: + - name: specificHumidity + where: + - variable: PreQC/specificHumidity + is_in: 9-15 + action: + name: reject + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 8061 diff --git a/parm/jcb-gdas/observations/atmosphere/aircraft_temperature.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/aircraft_temperature.yaml.j2 new file mode 100644 index 000000000..1c50f9064 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/aircraft_temperature.yaml.j2 @@ -0,0 +1,298 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: Aircraft + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsgrouping: + group variables: [stationIdentification] + sort variable: pressure + sort order: descending + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [airTemperature] + + # Observation Operator + # -------------------- + obs operator: + name: Composite + components: + - name: VertInterp + variables: [airTemperature] + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: Composite + components: + - name: VertInterp + variables: [airTemperature] + + # Observation Bias Correction (VarBC) + # ----------------------------------- + obs bias: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_acft_suffix}}" + output file: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_acft_suffix}}" + bc by record: true + variational bc: + predictors: + - name: constant + - name: obsMetadataPredictor + variable: instantaneousAltitudeRate + - name: obsMetadataPredictor + variable: instantaneousAltitudeRate + order: 2 + covariance: + minimal required obs number: 3 + variance range: [1.0e-6, 1.0] + step size: 1.0e-4 + largest analysis variance: 10000.0 + prior: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiascovin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovin_acft_suffix}}" + inflation: + ratio: 1.005 + ratio for small dataset: 2.0 + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_acft_suffix}}" + + + obs pre filters: + + # Error assignments + # Assign the initial observation error, based on pressure (for AIREP/ACARS; itype=130) + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [100000, 95000, 90000, 85000, 80000] + errors: [2.5, 2.3, 2.1, 1.9, 1.7] + where: + - variable: + name: ObsType/airTemperature + is_in: 130 + + # Assign the initial observation error, based on pressure (for AMDAR and MDCRS; itype=131,133) + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [100000, 95000, 90000, 85000, 80000] + errors: [1.4706, 1.3529, 1.2353, 1.1176, 1.0] + where: + - variable: + name: ObsType/airTemperature + is_in: 131,133 + + # Assign the initial observation error, based on pressure (for RECON aircraft; itype=132) + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000, 100000, 95000, 90000, 85000, 35000, 30000, 25000, 20000, 15000, 10000, 7500, 5000, 4000, 3000, 2000, 1000] + errors: [ 1.2, 1.2, 1.1, 0.9, 0.8, 0.8, 0.9, 1.2, 1.2, 1.0, 0.8, 0.8, 0.9, 0.95, 1.0, 1.25, 1.5] + where: + - variable: + name: ObsType/airTemperature + is_in: 132 + + # Assign the initial observation error, based on pressure (for TAMDAR itype=134) + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [100000, 95000, 90000, 85000, 80000, 60000, 40000, 7500] + errors: [1.5, 1.35, 1.25, 1.10, 1.0, 1.3, 1.7, 1000000000.0 ] + where: + - variable: + name: ObsType/airTemperature + is_in: 134 + + # Assign the initial observation error, based on pressure (for Canadian AMDAR itype=135) + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [100000, 95000, 90000, 85000, 80000] + errors: [1.4706, 1.3529, 1.2353, 1.1176, 1.0 ] + where: + - variable: + name: ObsType/airTemperature + is_in: 135 + +# Future section to re-assign kx0 here + + # Reassign any AIREP/PIREP (type=130) airTemperature MetaData/stationIdentification to 'KX130 ' + - filter: Variable Assignment + where: + - variable: + name: ObsType/airTemperature + is_in: 130 + assignments: + - name: MetaData/stationIdentification + value: "KX130" + + # Error inflation where QC is 3 or 7 + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: inflate error + inflation factor: 1.2 + where: + - variable: PreQC/airTemperature + is_in: 3,7 + + # Error inflation when observation pressure < 100 mb (read_prepbufr.f90) + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: inflate error + inflation factor: 1.2 + where: + - variable: MetaData/pressure + maxvalue: 10000.0 + + # Inflate error by 10 if kx==130 and P >=500 mb (read_prepbufr.f90) + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: inflate error + inflation factor: 10.0 + where: + - variable: ObsType/airTemperature + is_in: 130.0 + - variable: MetaData/pressure + minvalue: 50000.0 + + # Inflate if IALR > 30 + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: inflate error + inflation factor: 1.5 + where: + - variable: MetaData/instantaneousAltitudeRate + minvalue: 30.0 + + + obs prior filters: + + # error inflation based on pressure check + - filter: Perform Action + filter variables: + - name: airTemperature + where: + - variable: + name: ObsType/airTemperature + is_in: 130-135 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: airTemperature + inflation factor: 8.0 + surface_obs: true + geovar_sfc_geomz: height_above_mean_sea_level_at_surface + defer to post: true + + + obs post filters: + + - filter: Variable Assignment + assignments: + - name: HofXBc/airTemperature + type: float + function: + name: ObsFunction/LinearCombination + options: + variables: [HofX/airTemperature] + coefs: [1.0] + defer to post: true + + # Background check + # ratio threshold 7.0 + - filter: Background Check + test_hofx: HofXBc + filter variables: + - name: airTemperature + threshold: 7.0 + action: + name: reject + where: + - variable: PreQC/airTemperature + is_not_in: [3] + defer to post: true + + # ratio threshold * 0.7 if PreQC=3 + - filter: Background Check + test_hofx: HofXBc + filter variables: + - name: airTemperature + threshold: 4.9 + action: + name: reject + where: + - variable: PreQC/airTemperature + is_in: [3] + defer to post: true + + # Reject where QC = 4-19 + - filter: Perform Action + filter variables: + - name: airTemperature + where: + - variable: PreQC/airTemperature + is_in: 4-19 + action: + name: reject + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 114568 diff --git a/parm/jcb-gdas/observations/atmosphere/aircraft_wind.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/aircraft_wind.yaml.j2 new file mode 100644 index 000000000..a2ef0e54b --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/aircraft_wind.yaml.j2 @@ -0,0 +1,289 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: Aircraft + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsgrouping: + group variables: [stationIdentification] + sort variable: pressure + sort order: descending + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + # Observation Operator + # -------------------- + obs operator: + name: Composite + components: + - name: VertInterp + variables: + - name: windEastward + - name: windNorthward + # Hofx scaling + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: Composite + components: + - name: VertInterp + variables: + - name: windEastward + - name: windNorthward + + # Hofx scaling + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + + obs pre filters: + # Assign intial ObsError specific to AIREP/ACARS + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error parameter: 3.6 + where: + - variable: + name: ObsType/windEastward + is_in: 230 + + # Assign intial ObsError specific to AMDAR + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error parameter: 3.0 + where: + - variable: + name: ObsType/windEastward + is_in: 231 + + # Assign intial ObsError specific to MDCRS + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error parameter: 2.5 + where: + - variable: + name: ObsType/windEastward + is_in: 233 + + + # Assign intial ObsError specific to TAMDAR, Canadian AMDAR + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error parameter: 3.0 + where: + - variable: + name: ObsType/windEastward + is_in: 234, 235 + + # Assign the initial ObsError, based on height/pressure for RECON aircraft + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000, 70000, 65000, 60000, 55000, 50000, 45000, 40000, 35000, 30000, 25000, 20000, 15000, 10000, 7500, 5000] + errors: [2.4, 2.4, 2.5, 2.6, 2.7, 2.8, 2.95, 3.1, 3.25, 3.4, 3.175, 2.95, 2.725, 2.5, 2.6, 2.7] + where: + - variable: + name: ObsType/windEastward + is_in: 232 + + # Error inflation when QC is 3 or 7 + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: inflate error + inflation factor: 1.2 + where: + - variable: PreQC/windNorthward + is_in: 3,7 + + # Error inflation when observation pressure < 50 hPa (read_prepbufr.f90) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: inflate error + inflation factor: 1.2 + where: + - variable: MetaData/pressure + maxvalue: 5000.0 + + + obs prior filters: + + # Apply variable changes needed for performing wind scaling + # --------------------------------------------------------- + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: True + + + # Error inflation based on pressure check + - filter: Perform Action + filter variables: + - name: windEastward + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windEastward + inflation factor: 4.0 + surface_obs: false + geovar_sfc_geomz: height_above_mean_sea_level_at_surface + defer to post: true + + # Error inflation based on pressure check + - filter: Perform Action + filter variables: + - name: windNorthward + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + surface_obs: false + #test_obserr: GsiAdjustObsError + geovar_sfc_geomz: height_above_mean_sea_level_at_surface + defer to post: true + + + obs post filters: + + # Gross Check + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 230, 231, 232, 233, 234, 235 ] + cgross: [ 6.0, 6.5, 7.0, 7.5, 7.5, 7.5 ] + error_min: [ 1.4, 1.4, 1.4, 1.4, 1.4, 1.4 ] + error_max: [ 6.1, 6.1, 6.1, 6.1, 6.1, 6.1 ] + variable: windEastward + where: + - variable: PreQC/windEastward + is_not_in: 3 + action: + name: reject + defer to post: true + + # Gross Check + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 230, 231, 232, 233, 234, 235 ] + cgross: [ 4.2, 4.55, 4.9, 5.25, 5.25, 5.25 ] + error_min: [ 1.4, 1.4, 1.4, 1.4, 1.4, 1.4 ] + error_max: [ 6.1, 6.1, 6.1, 6.1, 6.1, 6.1 ] + variable: windEastward + where: + - variable: PreQC/windEastward + is_in: 3 + action: + name: reject + defer to post: true + + # Gross Check + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 230, 231, 232, 233, 234, 235 ] + cgross: [ 6.0, 6.5, 7.0, 7.5, 7.5, 7.5 ] + error_min: [ 1.4, 1.4, 1.4, 1.4, 1.4, 1.4 ] + error_max: [ 6.1, 6.1, 6.1, 6.1, 6.1, 6.1 ] + variable: windNorthward + where: + - variable: PreQC/windNorthward + is_not_in: 3 + action: + name: reject + defer to post: true + + # Gross Check + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 230, 231, 232, 233, 234, 235 ] + cgross: [ 4.2, 4.55, 4.9, 5.25, 5.25, 5.25 ] + error_min: [ 1.4, 1.4, 1.4, 1.4, 1.4, 1.4 ] + error_max: [ 6.1, 6.1, 6.1, 6.1, 6.1, 6.1 ] + variable: windNorthward + where: + - variable: PreQC/windNorthward + is_in: 3 + action: + name: reject + defer to post: true + + # Reject where QC is 9-15 + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: PreQC/windNorthward + is_in: 9-15 + action: + name: reject + defer to post: true + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 227312 diff --git a/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml b/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml index 8425b80f6..3ac63c49b 100644 --- a/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml +++ b/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml @@ -104,16 +104,20 @@ atmosphere_obsdataout_suffix: "_2024020100.nc" atmosphere_obsbiasin_path: DATA/obs atmosphere_obsbiasin_prefix: GPREFIX atmosphere_obsbiasin_suffix: ".satbias.nc" +atmosphere_obsbiasin_acft_suffix: ".acft_out.nc" atmosphere_obstlapsein_prefix: GPREFIX atmosphere_obstlapsein_suffix: ".tlapse.txt" atmosphere_obsbiascovin_prefix: GPREFIX atmosphere_obsbiascovin_suffix: ".satbias_cov.nc" +atmosphere_obsbiascovin_acft_suffix: ".acft_out_cov.nc" atmosphere_obsbiasout_path: DATA/bc atmosphere_obsbiasout_prefix: APREFIX atmosphere_obsbiasout_suffix: ".satbias.nc" +atmosphere_obsbiasout_acft_suffix: ".acft_out.nc" atmosphere_obsbiascovout_prefix: APREFIX atmosphere_obsbiascovout_suffix: ".satbias_cov.nc" +atmosphere_obsbiascovout_acft_suffix: ".acft_out_cov.nc" # Local Ensemble DA (LETKF) From 8d74c82cbbf00023aedf1ed2426fefc87877b47c Mon Sep 17 00:00:00 2001 From: Anna Shlyaeva Date: Mon, 30 Jun 2025 09:05:07 -0600 Subject: [PATCH 134/199] Add yaml section for truncating increments to a certain precision (#145) All in the title. Required for reproducibility. --------- Co-authored-by: Anna Shlyaeva --- .../jcb-gdas/algorithm/marine/soca_incpostproc.yaml.j2 | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/parm/jcb-gdas/algorithm/marine/soca_incpostproc.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_incpostproc.yaml.j2 index 613820cf8..a2f0c2ab9 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_incpostproc.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_incpostproc.yaml.j2 @@ -42,6 +42,16 @@ output increment: type: incr output file: inc.nc +increment precision: +- field: sea_water_potential_temperature + precision: 1.e-7 +- field: sea_water_salinity + precision: 1.e-7 +- field: sea_surface_height_above_geoid # Not used in MOM6 iau + precision: 1.e-7 +- field: sea_ice_area_fraction + precision: 1.e-7 + qc increment: state bounds: sea_water_potential_temperature: [-2.5, 36.0] From 6775e25175b58dda8ecff9ece039dc566d975737 Mon Sep 17 00:00:00 2001 From: yuanxue2870 <136842224+yuanxue2870@users.noreply.github.com> Date: Tue, 8 Jul 2025 14:45:26 -0400 Subject: [PATCH 135/199] Replace slmsk with fraction_of_land (#146) In order to solve https://github.com/NOAA-EMC/GFS/issues/8 (jcb-gdas repo subissue: https://github.com/NOAA-EMC/jcb-gdas/issues/144; fv3-jedi repo subissue: https://github.com/JCSDA-internal/fv3-jedi/issues/1386), we need to 1) add capability to read fraction_of_land from orog file; and 2) replace slmsk with fraction_of_land (minvalue: 0.0001; the cutoff value is carefully determined based on analysis between land_frac and slmsk obtained from latest restart files to mimic the impacts when using slmsk=1). This PR requires dependency on a fv3-jedi (JCSDA-internal) PR: https://github.com/JCSDA-internal/fv3-jedi/pull/1387. G-W CI tests are done to test these changes (G-W repo issue: https://github.com/NOAA-EMC/global-workflow/issues/3837). All is passed. --------- Co-authored-by: Jiarui Dong --- .../snow/snow_3dvar_outer_loop_1.yaml.j2 | 2 + .../model/snow/snow_background.yaml.j2 | 2 + .../observations/snow/adpsfc_snow.yaml.j2 | 5 +- .../observations/snow/ghcn_snow.yaml.j2 | 5 +- .../observations/snow/ims_snow.yaml.j2 | 5 +- .../observations/snow/madis_snow.yaml.j2 | 5 +- .../jcb-gdas/observations/snow/sfcsno.yaml.j2 | 5 +- .../jcb-gdas/observations/snow/snocvr.yaml.j2 | 5 +- .../observations/snow/snocvr_snow.yaml.j2 | 242 ------------------ 9 files changed, 16 insertions(+), 260 deletions(-) delete mode 100644 parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 diff --git a/parm/jcb-gdas/model/snow/snow_3dvar_outer_loop_1.yaml.j2 b/parm/jcb-gdas/model/snow/snow_3dvar_outer_loop_1.yaml.j2 index 07aacc206..53085b711 100644 --- a/parm/jcb-gdas/model/snow/snow_3dvar_outer_loop_1.yaml.j2 +++ b/parm/jcb-gdas/model/snow/snow_3dvar_outer_loop_1.yaml.j2 @@ -19,8 +19,10 @@ skip coupler file: true state variables: - filtered_orography + - fraction_of_land field io names: filtered_orography: orog_filt + fraction_of_land: land_frac datapath: {{ snow_orog_files_path }}/ filename_orog: {{ snow_orog_prefix }}_oro_data.nc diagnostics: diff --git a/parm/jcb-gdas/model/snow/snow_background.yaml.j2 b/parm/jcb-gdas/model/snow/snow_background.yaml.j2 index 79630fb94..2c718220b 100644 --- a/parm/jcb-gdas/model/snow/snow_background.yaml.j2 +++ b/parm/jcb-gdas/model/snow/snow_background.yaml.j2 @@ -9,10 +9,12 @@ state variables: - sheleg - filtered_orography - fraction_of_ice +- fraction_of_land field io names: totalSnowDepth: snodl filtered_orography: orog_filt fraction_of_ice: fice + fraction_of_land: land_frac filename_sfcd: '{{ snow_background_time_fv3 }}.sfc_data.nc' filename_cplr: '{{ snow_background_time_fv3 }}.coupler.res' filename_orog: '{{ snow_orog_prefix }}_oro_data.nc' diff --git a/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 index 4cb4b3306..1d7c8a2b0 100644 --- a/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 @@ -115,9 +115,8 @@ - filter: Domain Check # land only where: - variable: - name: GeoVaLs/slmsk - minvalue: 0.5 - maxvalue: 1.5 + name: GeoVaLs/fraction_of_land + minvalue: 0.0001 actions: - name: set flag: land_check diff --git a/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 index b825e7937..e83755acb 100644 --- a/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 @@ -37,9 +37,8 @@ - filter: Domain Check # land only where: - variable: - name: GeoVaLs/slmsk - minvalue: 0.5 - maxvalue: 1.5 + name: GeoVaLs/fraction_of_land + minvalue: 0.0001 - filter: RejectList # no land-ice where: - variable: diff --git a/parm/jcb-gdas/observations/snow/ims_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/ims_snow.yaml.j2 index fd7ca7ba2..72e22eea2 100644 --- a/parm/jcb-gdas/observations/snow/ims_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/ims_snow.yaml.j2 @@ -38,9 +38,8 @@ - filter: Domain Check # land only where: - variable: - name: GeoVaLs/slmsk - minvalue: 0.5 - maxvalue: 1.5 + name: GeoVaLs/fraction_of_land + minvalue: 0.0001 - filter: Domain Check # land only, no sea ice where: - variable: diff --git a/parm/jcb-gdas/observations/snow/madis_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/madis_snow.yaml.j2 index c0120105d..0f41bab1b 100644 --- a/parm/jcb-gdas/observations/snow/madis_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/madis_snow.yaml.j2 @@ -114,9 +114,8 @@ - filter: Domain Check # land only where: - variable: - name: GeoVaLs/slmsk - minvalue: 0.5 - maxvalue: 1.5 + name: GeoVaLs/fraction_of_land + minvalue: 0.0001 actions: - name: set flag: land_check diff --git a/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 b/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 index d51d0d5ac..8b65b8a7a 100644 --- a/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 @@ -115,9 +115,8 @@ - filter: Domain Check # land only where: - variable: - name: GeoVaLs/slmsk - minvalue: 0.5 - maxvalue: 1.5 + name: GeoVaLs/fraction_of_land + minvalue: 0.0001 actions: - name: set flag: land_check diff --git a/parm/jcb-gdas/observations/snow/snocvr.yaml.j2 b/parm/jcb-gdas/observations/snow/snocvr.yaml.j2 index d5ac5b6af..d5bc83050 100644 --- a/parm/jcb-gdas/observations/snow/snocvr.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/snocvr.yaml.j2 @@ -115,9 +115,8 @@ - filter: Domain Check # land only where: - variable: - name: GeoVaLs/slmsk - minvalue: 0.5 - maxvalue: 1.5 + name: GeoVaLs/fraction_of_land + minvalue: 0.0001 actions: - name: set flag: land_check diff --git a/parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 deleted file mode 100644 index 144ecbbde..000000000 --- a/parm/jcb-gdas/observations/snow/snocvr_snow.yaml.j2 +++ /dev/null @@ -1,242 +0,0 @@ -- - - # Observation Space (I/O) - # ----------------------- - obs space: - name: snocvr_snow - obsdatain: - engine: - type: H5File - obsfile: "{{snow_obsdatain_path}}/{{snow_obsdatain_prefix}}{{observation_from_jcb}}.nc4" - missing file action: warn - obsgrouping: - group variables: [stationIdentification] - obsdataout: - engine: - type: H5File - obsfile: "{{snow_obsdataout_path}}/{{snow_obsdataout_prefix}}{{observation_from_jcb}}{{snow_obsdataout_suffix}}" - simulated variables: [totalSnowDepth] - # - - # Observation Operator - # -------------------- - obs operator: - name: Composite - components: - - name: Identity - - name: BackgroundErrorIdentity - linear obs operator: - name: Identity - # - - # Observation Filters (QC) - # ------------------------ - obs pre filters: - - filter: Create Diagnostic Flags - flags: - - name: missing_snowdepth - initial value: false - - name: missing_elevation - initial value: false - - name: temporal_thinning - initial value: false - - name: invalid_snowdepth - initial value: false - - name: invalid_elevation - initial value: false - - name: land_check - initial value: false - - name: landice_check - initial value: false - - name: elevation_bkgdiff - initial value: false - - name: background_check - initial value: false - - name: buddy_check - initial value: false - - filter: Perform Action - filter variables: - - name: totalSnowDepth - action: - name: assign error - error parameter: 40.0 - - filter: Variable Assignment - assignments: - - name: GrossErrorProbability/totalSnowDepth - type: float - value: 0.02 - - name: BkgError/totalSnowDepth_background_error - type: float - value: 30.0 - - filter: Domain Check - where: - - variable: - name: ObsValue/totalSnowDepth - value: is_valid - actions: - - name: set - flag: missing_snowdepth - ignore: rejected observations - - name: reject - - filter: Domain Check - where: - - variable: - name: MetaData/stationElevation - value: is_valid - actions: - - name: set - flag: missing_elevation - ignore: rejected observations - - name: reject - - filter: Temporal Thinning - min_spacing: '{{ window_length }}' - seed_time: '{{ snow_background_time_iso }}' - category_variable: - name: MetaData/stationIdentification - actions: - - name: set - flag: temporal_thinning - ignore: rejected observations - - name: reject - obs prior filters: - - filter: Bounds Check - filter variables: - - name: totalSnowDepth - minvalue: 0.0 - maxvalue: 20000.0 - actions: - - name: set - flag: invalid_snowdepth - ignore: rejected observations - - name: reject - - filter: Domain Check # land only - where: - - variable: - name: GeoVaLs/slmsk - minvalue: 0.5 - maxvalue: 1.5 - actions: - - name: set - flag: land_check - ignore: rejected observations - - name: reject - - filter: Domain Check - where: - - variable: - name: MetaData/stationElevation - minvalue: -200.0 - maxvalue: 9900.0 - actions: - - name: set - flag: invalid_elevation - ignore: rejected observations - - name: reject - - filter: Domain Check # land only, no sea ice - where: - - variable: - name: GeoVaLs/fraction_of_ice - maxvalue: 0.0 - - filter: RejectList # no land-ice - where: - - variable: - name: GeoVaLs/vtype - minvalue: 14.5 - maxvalue: 15.5 - actions: - - name: set - flag: landice_check - ignore: rejected observations - - name: reject - - filter: Difference Check # elevation check - reference: MetaData/stationElevation - value: GeoVaLs/filtered_orography - threshold: 800. - actions: - - name: set - flag: elevation_bkgdiff - ignore: rejected observations - - name: reject - # Add inflate error characterized by a function of elevation difference - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/elevation_diff - type: float - function: - name: ObsFunction/Arithmetic - options: - variables: [MetaData/stationElevation, GeoVaLs/filtered_orography] - coefficients: [1., -1.] - total exponent: 2 - total coefficient: 0.0000015625 #1/(800*800) - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/elevation_diff_exp - type: float - function: - name: ObsFunction/Exponential - options: - variables: [DerivedMetaData/elevation_diff] - coeffA: 1. - coeffB: -1. - coeffC: 0. - coeffD: 1. - - filter: Variable Assignment - assignments: - - name: DerivedMetaData/elevation_diff_exp_inv - type: float - function: - name: ObsFunction/Arithmetic - options: - variables: [DerivedMetaData/elevation_diff_exp] - total exponent: -1. - - filter: Perform Action - filter variables: - - name: totalSnowDepth - where: - - variable: - name: ObsValue/totalSnowDepth - minvalue: 0. - action: - name: inflate error - inflation variable: DerivedMetaData/elevation_diff_exp_inv # 2.0 - obs post filters: - - filter: Background Check # gross error check - filter variables: - - name: totalSnowDepth - threshold: 6.25 - absolute threshold: 250 - bias correction parameter: 1.0 - actions: - - name: set - flag: background_check - ignore: rejected observations - - name: reject - - filter: Met Office Buddy Check - filter variables: - - name: totalSnowDepth - rejection_threshold: 0.5 - traced_boxes: # trace all observations - min_latitude: -90 - max_latitude: 90 - min_longitude: -180 - max_longitude: 180 - search_radius: 150 # km - station_id_variable: - name: MetaData/stationIdentification - num_zonal_bands: 24 - sort_by_pressure: false - max_total_num_buddies: 15 - max_num_buddies_from_single_band: 10 - max_num_buddies_with_same_station_id: 5 - use_legacy_buddy_collector: false - horizontal_correlation_scale: { "-90": 150, "90": 150 } - temporal_correlation_scale: PT6H - damping_factor_1: 1.0 - damping_factor_2: 1.0 - background_error_group: BkgError - actions: - - name: set - flag: buddy_check - ignore: rejected observations - - name: reject - From 3b5147c9d43edec75002a7f8bc2ee7a43bae2107 Mon Sep 17 00:00:00 2001 From: JohnSteffen-NOAA <77806417+JohnSteffen-NOAA@users.noreply.github.com> Date: Tue, 15 Jul 2025 17:34:31 -0400 Subject: [PATCH 136/199] Add QC yamls for separate ADT retrievals (#151) For now, the individual ADT retrievals are soft linked to the adt_rads_all.yaml.j2 file for the QC. --- parm/jcb-gdas/observations/marine/adt_rads_cryosat2.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/adt_rads_jason3.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/adt_rads_saral.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/adt_rads_sentinel3a.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/adt_rads_sentinel3b.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/adt_rads_sentinel6a.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/adt_rads_swot.yaml.j2 | 1 + 7 files changed, 7 insertions(+) create mode 120000 parm/jcb-gdas/observations/marine/adt_rads_cryosat2.yaml.j2 create mode 120000 parm/jcb-gdas/observations/marine/adt_rads_jason3.yaml.j2 create mode 120000 parm/jcb-gdas/observations/marine/adt_rads_saral.yaml.j2 create mode 120000 parm/jcb-gdas/observations/marine/adt_rads_sentinel3a.yaml.j2 create mode 120000 parm/jcb-gdas/observations/marine/adt_rads_sentinel3b.yaml.j2 create mode 120000 parm/jcb-gdas/observations/marine/adt_rads_sentinel6a.yaml.j2 create mode 120000 parm/jcb-gdas/observations/marine/adt_rads_swot.yaml.j2 diff --git a/parm/jcb-gdas/observations/marine/adt_rads_cryosat2.yaml.j2 b/parm/jcb-gdas/observations/marine/adt_rads_cryosat2.yaml.j2 new file mode 120000 index 000000000..02ad0a9d7 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/adt_rads_cryosat2.yaml.j2 @@ -0,0 +1 @@ +adt_rads_all.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/adt_rads_jason3.yaml.j2 b/parm/jcb-gdas/observations/marine/adt_rads_jason3.yaml.j2 new file mode 120000 index 000000000..02ad0a9d7 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/adt_rads_jason3.yaml.j2 @@ -0,0 +1 @@ +adt_rads_all.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/adt_rads_saral.yaml.j2 b/parm/jcb-gdas/observations/marine/adt_rads_saral.yaml.j2 new file mode 120000 index 000000000..02ad0a9d7 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/adt_rads_saral.yaml.j2 @@ -0,0 +1 @@ +adt_rads_all.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/adt_rads_sentinel3a.yaml.j2 b/parm/jcb-gdas/observations/marine/adt_rads_sentinel3a.yaml.j2 new file mode 120000 index 000000000..02ad0a9d7 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/adt_rads_sentinel3a.yaml.j2 @@ -0,0 +1 @@ +adt_rads_all.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/adt_rads_sentinel3b.yaml.j2 b/parm/jcb-gdas/observations/marine/adt_rads_sentinel3b.yaml.j2 new file mode 120000 index 000000000..02ad0a9d7 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/adt_rads_sentinel3b.yaml.j2 @@ -0,0 +1 @@ +adt_rads_all.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/adt_rads_sentinel6a.yaml.j2 b/parm/jcb-gdas/observations/marine/adt_rads_sentinel6a.yaml.j2 new file mode 120000 index 000000000..02ad0a9d7 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/adt_rads_sentinel6a.yaml.j2 @@ -0,0 +1 @@ +adt_rads_all.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/adt_rads_swot.yaml.j2 b/parm/jcb-gdas/observations/marine/adt_rads_swot.yaml.j2 new file mode 120000 index 000000000..02ad0a9d7 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/adt_rads_swot.yaml.j2 @@ -0,0 +1 @@ +adt_rads_all.yaml.j2 \ No newline at end of file From 4191a94e9faaec6fed33e7240887b7e55e18b97a Mon Sep 17 00:00:00 2001 From: DavidNew-NOAA <134300700+DavidNew-NOAA@users.noreply.github.com> Date: Tue, 22 Jul 2025 08:36:26 -0400 Subject: [PATCH 137/199] Companion for GW refactoring of marine ensemble recentering (#152) This PR is a companion of NOAA-EMC/global-workflow#3882 and NOAA-EMC/gdasapp#1803 which refactor the the marine ensemble recentering script and moves it into Global Workflow from GDASApp. The only change in this repository is the creation of a JCB template for the soca ensemble handler application --- .../algorithm/marine/soca_ens_handler.yaml.j2 | 125 ++++++++++++++++++ 1 file changed, 125 insertions(+) create mode 100644 parm/jcb-gdas/algorithm/marine/soca_ens_handler.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/marine/soca_ens_handler.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_ens_handler.yaml.j2 new file mode 100644 index 000000000..99a112fe3 --- /dev/null +++ b/parm/jcb-gdas/algorithm/marine/soca_ens_handler.yaml.j2 @@ -0,0 +1,125 @@ +# This yaml is for applying deterministic recentering increments to the ensemble members + +geometry: + mom6_input_nml: mom_input.nml + fields metadata: ./fields_metadata.yaml + +date: "{{ marine_window_end_iso }}" + +layers variable: [sea_water_cell_thickness] + +# TODO(AFE) fix ice recentering in cycled da +increment variables: +- sea_water_potential_temperature +- sea_water_salinity +- eastward_sea_water_velocity +- northward_sea_water_velocity +- sea_ice_area_fraction +- sea_ice_thickness +- sea_ice_snow_thickness +set increment variables to zero: +- eastward_sea_water_velocity +- northward_sea_water_velocity + +vertical geometry: + date: "{{ marine_window_begin_iso }}" + basename: ./INPUT/ + ocn_filename: MOM.res.nc + read_from_file: 3 + +add recentering increment: true +recentering around deterministic: true +sea ice recenter: true +sea ice analysis: + pattern: "%mem%" + date: "{{ marine_window_end_iso }}" + ocn_filename: "ocean.%mem%.nc" + ice_filename: "ice.%mem%.nc" + read_from_file: 1 + basename: {{ enspert_relpath }}/ens/ + state variables: + - sea_water_potential_temperature + - sea_water_salinity + - eastward_sea_water_velocity + - northward_sea_water_velocity + - sea_water_cell_thickness + - sea_water_depth + - ocean_mixed_layer_thickness + - sea_ice_area_fraction + - sea_ice_thickness + - sea_ice_snow_thickness + +sea ice variable change: + variable change name: Soca2Cice + arctic: + seaice edge: 0.4 + shuffle: true + rescale prior: + rescale: true + min hice: 0.5 + min hsno: 0.1 + antarctic: + seaice edge: 0.4 + shuffle: true + rescale prior: + rescale: true + min hice: 0.5 + min hsno: 0.1 + cice background state: + pattern: "%mem%" + restart: ens/cice_model.res.%mem%.nc + ncat: 5 + ice_lev: 7 + sno_lev: 1 + cice output: + pattern: "%mem%" + restart: ens/cice_model.res.output.%mem%.nc + output variables: + - sea_water_potential_temperature + - sea_water_salinity + - sea_ice_area_fraction + - sea_ice_thickness + - sea_ice_snow_thickness + +soca increments: # Could also be states, but they are read as increments + number of increments: '{{ marine_number_ensemble_members }}' + pattern: "%mem%" + template: + date: "{{ marine_window_end_iso }}" + basename: ./ens/ + ocn_filename: "ocean.%mem%.nc" + ice_filename: "ice.%mem%.nc" + read_from_file: 3 + +trajectory: +# TODO(AFE) fix ice recentering in cycled da + state variables: + - sea_water_potential_temperature + - sea_water_salinity + - eastward_sea_water_velocity + - northward_sea_water_velocity + - sea_water_cell_thickness + - sea_water_depth + - ocean_mixed_layer_thickness + - sea_ice_area_fraction + - sea_ice_thickness + - sea_ice_snow_thickness + date: "{{ marine_window_end_iso }}" + basename: ./bkg/ + ocn_filename: ocean.bkg.f009.nc + ice_filename: ice.bkg.f009.nc + read_from_file: 1 + +output increment: + datadir: ./ + date: "{{ marine_window_end_iso }}" + exp: trash + type: incr + output file: "recenter.incr.%mem%.nc" + pattern: "%mem%" + +ensemble variance output: + datadir: ./ + date: "{{ marine_window_end_iso }}" + exp: ensvar + type: incr From ec99d49744dd2c20004cb941da79554867fcc4ff Mon Sep 17 00:00:00 2001 From: Anna Shlyaeva Date: Fri, 25 Jul 2025 08:42:11 -0600 Subject: [PATCH 138/199] Add thinning for ice obs when used in LETKF (#153) Thin AMSR2 ice concentration obs, only for LETKF to avoid messing with gfsv17 configuration. I ran a test and confirmed that obs are thinned in LETKF, and not thinned in Var. Note: requires gdas PR to work correctly. Co-authored-by: Anna Shlyaeva --- .../observations/marine/icec_amsr2_north.yaml.j2 | 14 ++++++++------ .../observations/marine/icec_amsr2_south.yaml.j2 | 14 ++++++++------ 2 files changed, 16 insertions(+), 12 deletions(-) diff --git a/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 index 50770cf00..1dd133033 100644 --- a/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 @@ -40,12 +40,14 @@ action: name: inflate error inflation factor: 2.0 -# - filter: Gaussian Thinning -# horizontal_mesh: 25 #km -# use_reduced_horizontal_grid: true -# select_median: true -# action: -# name: reduce obs space +{% if letkf_app | default(false) %} + - filter: Gaussian Thinning + horizontal_mesh: 25 #km + use_reduced_horizontal_grid: true + select_median: true + action: + name: reduce obs space +{% endif %} {% if letkf_app | default(false) %} obs localizations: - localization method: Rossby diff --git a/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 index 50770cf00..1dd133033 100644 --- a/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 @@ -40,12 +40,14 @@ action: name: inflate error inflation factor: 2.0 -# - filter: Gaussian Thinning -# horizontal_mesh: 25 #km -# use_reduced_horizontal_grid: true -# select_median: true -# action: -# name: reduce obs space +{% if letkf_app | default(false) %} + - filter: Gaussian Thinning + horizontal_mesh: 25 #km + use_reduced_horizontal_grid: true + select_median: true + action: + name: reduce obs space +{% endif %} {% if letkf_app | default(false) %} obs localizations: - localization method: Rossby From aeaa5367c62b9a666ab30891876e76ef8c5fc9e8 Mon Sep 17 00:00:00 2001 From: Jiarui Dong Date: Mon, 28 Jul 2025 16:15:19 -0400 Subject: [PATCH 139/199] Remove buddy check filter and remove threshold option from background check filter for snow DA (#150) This PR removes the buddy check filter and the associated buddy check flag. This PR removes the threshold option from the background check filter. This PR also include the updates #148 from @yuanxue2870 to change the fraction_of_land threshold to 0.5. Resolves #147 and #149 --------- Co-authored-by: yuanxue2870 <136842224+yuanxue2870@users.noreply.github.com> Co-authored-by: Cory Martin --- .../algorithm/snow/snow_obs_staging.yaml.j2 | 2 + .../snow/t00z/ghcn_snow.yaml | 14 ++ .../snow/t06z/ghcn_snow.yaml | 14 ++ .../snow/t06z/snocvr.yaml | 2 +- .../observations/snow/adpsfc_snow.yaml.j2 | 105 ++++++----- .../observations/snow/ghcn_snow.yaml.j2 | 163 +++++++++++++++++- .../observations/snow/ims_snow.yaml.j2 | 51 +++++- .../observations/snow/madis_snow.yaml.j2 | 53 +----- .../jcb-gdas/observations/snow/sfcsno.yaml.j2 | 55 +----- .../jcb-gdas/observations/snow/snocvr.yaml.j2 | 53 +----- 10 files changed, 316 insertions(+), 196 deletions(-) create mode 100644 parm/jcb-gdas/observation_chronicle/snow/t00z/ghcn_snow.yaml create mode 100644 parm/jcb-gdas/observation_chronicle/snow/t06z/ghcn_snow.yaml diff --git a/parm/jcb-gdas/algorithm/snow/snow_obs_staging.yaml.j2 b/parm/jcb-gdas/algorithm/snow/snow_obs_staging.yaml.j2 index f50b5a911..17703ebd2 100644 --- a/parm/jcb-gdas/algorithm/snow/snow_obs_staging.yaml.j2 +++ b/parm/jcb-gdas/algorithm/snow/snow_obs_staging.yaml.j2 @@ -8,6 +8,8 @@ copy_opt: {% if observation_from_jcb == 'madis_snow' %} # madis_snow is a .nc file with filename 'snocvr_snow' in global GDA. - ['{{snow_obsdataroot_path}}/{{snow_obsdatain_prefix}}snocvr_snow.nc4', '{{snow_obsdatain_path}}'] + {% elif observation_from_jcb == 'ghcn_snow' %} + - ['{{snow_obsdataroot_path}}/{{snow_obsdatain_prefix}}ghcn_snow.nc', '{{snow_obsdatain_path}}'] {% else %} - ['{{snow_obsdataroot_path}}/{{snow_obsdatain_prefix}}{{observation_from_jcb}}{{snow_obsdatain_suffix}}', '{{snow_obsdatain_path}}'] {% endif %} diff --git a/parm/jcb-gdas/observation_chronicle/snow/t00z/ghcn_snow.yaml b/parm/jcb-gdas/observation_chronicle/snow/t00z/ghcn_snow.yaml new file mode 100644 index 000000000..f7db866a8 --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/snow/t00z/ghcn_snow.yaml @@ -0,0 +1,14 @@ +# Instrument metadata +# ------------------- +commissioned: 1970-01-01T00:00:00 + +observer_type: conventional # Type of chronicle to use + +window_option: max + +# observation type initial configuration +# -------------------------------- +stations_to_reject: [fakelist] + +# Chronicle of changes for this observation type +# ---------------------------------------- diff --git a/parm/jcb-gdas/observation_chronicle/snow/t06z/ghcn_snow.yaml b/parm/jcb-gdas/observation_chronicle/snow/t06z/ghcn_snow.yaml new file mode 100644 index 000000000..f7db866a8 --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/snow/t06z/ghcn_snow.yaml @@ -0,0 +1,14 @@ +# Instrument metadata +# ------------------- +commissioned: 1970-01-01T00:00:00 + +observer_type: conventional # Type of chronicle to use + +window_option: max + +# observation type initial configuration +# -------------------------------- +stations_to_reject: [fakelist] + +# Chronicle of changes for this observation type +# ---------------------------------------- diff --git a/parm/jcb-gdas/observation_chronicle/snow/t06z/snocvr.yaml b/parm/jcb-gdas/observation_chronicle/snow/t06z/snocvr.yaml index 222770c84..e0965b670 100644 --- a/parm/jcb-gdas/observation_chronicle/snow/t06z/snocvr.yaml +++ b/parm/jcb-gdas/observation_chronicle/snow/t06z/snocvr.yaml @@ -8,7 +8,7 @@ window_option: max # observation type initial configuration # -------------------------------- -stations_to_reject: [BFTU1, FRDN5, MWDO3, SGNN5, NUTA3, CNDA3, HBEA3, ELCN5] +stations_to_reject: [BFTU1, FRDN5, MWDO3, SGNN5, NUTA3, CNDA3, HBEA3, ELCN5] # Chronicle of changes for this observation type # ---------------------------------------- diff --git a/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 index 1d7c8a2b0..596dd3ffd 100644 --- a/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 @@ -22,11 +22,6 @@ # Observation Operator # -------------------- obs operator: - name: Composite - components: - - name: Identity - - name: BackgroundErrorIdentity - linear obs operator: name: Identity # @@ -49,28 +44,20 @@ initial value: false - name: landice_check initial value: false + - name: seaice_check + initial value: false - name: elevation_bkgdiff initial value: false - name: rejectlist initial value: false - name: background_check initial value: false - - name: buddy_check - initial value: false - filter: Perform Action filter variables: - name: totalSnowDepth action: name: assign error error parameter: 40.0 - - filter: Variable Assignment - assignments: - - name: GrossErrorProbability/totalSnowDepth - type: float - value: 0.02 - - name: BkgError/totalSnowDepth_background_error - type: float - value: 30.0 - filter: Domain Check where: - variable: @@ -116,7 +103,7 @@ where: - variable: name: GeoVaLs/fraction_of_land - minvalue: 0.0001 + minvalue: 0.5 actions: - name: set flag: land_check @@ -133,6 +120,16 @@ flag: invalid_elevation ignore: rejected observations - name: reject + - filter: Domain Check # land only, no sea ice + where: + - variable: + name: GeoVaLs/fraction_of_ice + maxvalue: 0.0 + actions: + - name: set + flag: seaice_check + ignore: rejected observations + - name: reject - filter: RejectList # no land-ice where: - variable: @@ -147,12 +144,55 @@ - filter: Difference Check # elevation check reference: MetaData/stationElevation value: GeoVaLs/filtered_orography - threshold: 200. + threshold: 800. actions: - name: set flag: elevation_bkgdiff ignore: rejected observations - name: reject + # Add inflate error characterized by a function of elevation difference + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/elevation_diff + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: [MetaData/stationElevation, GeoVaLs/filtered_orography] + coefficients: [1., -1.] + total exponent: 2 + total coefficient: 0.0000015625 #1/(800*800) + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/elevation_diff_exp + type: float + function: + name: ObsFunction/Exponential + options: + variables: [DerivedMetaData/elevation_diff] + coeffA: 1. + coeffB: -1. + coeffC: 0. + coeffD: 1. + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/elevation_diff_exp_inv + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: [DerivedMetaData/elevation_diff_exp] + total exponent: -1. + - filter: Perform Action + filter variables: + - name: totalSnowDepth + where: + - variable: + name: ObsValue/totalSnowDepth + minvalue: 0. + action: + name: inflate error + inflation variable: DerivedMetaData/elevation_diff_exp_inv # 2.0 - filter: BlackList where: - variable: @@ -167,38 +207,11 @@ - filter: Background Check # gross error check filter variables: - name: totalSnowDepth - threshold: 6.25 + absolute threshold: 250 + bias correction parameter: 1.0 actions: - name: set flag: background_check ignore: rejected observations - name: reject - - filter: Met Office Buddy Check - filter variables: - - name: totalSnowDepth - rejection_threshold: 0.5 - traced_boxes: # trace all observations - min_latitude: -90 - max_latitude: 90 - min_longitude: -180 - max_longitude: 180 - search_radius: 150 # km - station_id_variable: - name: MetaData/stationIdentification - num_zonal_bands: 24 - sort_by_pressure: false - max_total_num_buddies: 15 - max_num_buddies_from_single_band: 10 - max_num_buddies_with_same_station_id: 5 - use_legacy_buddy_collector: false - horizontal_correlation_scale: { "-90": 150, "90": 150 } - temporal_correlation_scale: PT6H - damping_factor_1: 1.0 - damping_factor_2: 1.0 - background_error_group: BkgError - actions: - - name: set - flag: buddy_check - ignore: rejected observations - - name: reject diff --git a/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 index e83755acb..f8fd5f839 100644 --- a/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 @@ -1,4 +1,4 @@ - - +- # Observation Space (I/O) # ----------------------- @@ -24,34 +24,179 @@ # Observation Filters (QC) # ------------------------ - obs filters: - - filter: Bounds Check + obs pre filters: + - filter: Create Diagnostic Flags + flags: + - name: missing_snowdepth + initial value: false + - name: missing_elevation + initial value: false + - name: invalid_snowdepth + initial value: false + - name: invalid_elevation + initial value: false + - name: land_check + initial value: false + - name: landice_check + initial value: false + - name: seaice_check + initial value: false + - name: elevation_bkgdiff + initial value: false + - name: rejectlist + initial value: false + - name: background_check + initial value: false + - filter: Perform Action filter variables: - name: totalSnowDepth - minvalue: 0.0 + action: + name: assign error + error parameter: 40.0 + - filter: Domain Check + where: + - variable: + name: ObsValue/totalSnowDepth + value: is_valid + actions: + - name: set + flag: missing_snowdepth + ignore: rejected observations + - name: reject - filter: Domain Check where: - variable: name: MetaData/stationElevation value: is_valid + actions: + - name: set + flag: missing_elevation + ignore: rejected observations + - name: reject + obs prior filters: + - filter: Bounds Check + filter variables: + - name: totalSnowDepth + minvalue: 0.0 + maxvalue: 20000.0 + actions: + - name: set + flag: invalid_snowdepth + ignore: rejected observations + - name: reject - filter: Domain Check # land only where: - variable: name: GeoVaLs/fraction_of_land - minvalue: 0.0001 + minvalue: 0.5 + actions: + - name: set + flag: land_check + ignore: rejected observations + - name: reject + - filter: Domain Check + where: + - variable: + name: MetaData/stationElevation + minvalue: -200.0 + maxvalue: 9900.0 + actions: + - name: set + flag: invalid_elevation + ignore: rejected observations + - name: reject + - filter: Domain Check # land only, no sea ice + where: + - variable: + name: GeoVaLs/fraction_of_ice + maxvalue: 0.0 + actions: + - name: set + flag: seaice_check + ignore: rejected observations + - name: reject - filter: RejectList # no land-ice where: - variable: name: GeoVaLs/vtype minvalue: 14.5 maxvalue: 15.5 + actions: + - name: set + flag: landice_check + ignore: rejected observations + - name: reject - filter: Difference Check # elevation check reference: MetaData/stationElevation value: GeoVaLs/filtered_orography - threshold: 200. - - filter: Background Check + threshold: 800. + actions: + - name: set + flag: elevation_bkgdiff + ignore: rejected observations + - name: reject + # Add inflate error characterized by a function of elevation difference + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/elevation_diff + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: [MetaData/stationElevation, GeoVaLs/filtered_orography] + coefficients: [1., -1.] + total exponent: 2 + total coefficient: 0.0000015625 #1/(800*800) + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/elevation_diff_exp + type: float + function: + name: ObsFunction/Exponential + options: + variables: [DerivedMetaData/elevation_diff] + coeffA: 1. + coeffB: -1. + coeffC: 0. + coeffD: 1. + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/elevation_diff_exp_inv + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: [DerivedMetaData/elevation_diff_exp] + total exponent: -1. + - filter: Perform Action filter variables: - name: totalSnowDepth - threshold: 6.25 + where: + - variable: + name: ObsValue/totalSnowDepth + minvalue: 0. action: - name: reject + name: inflate error + inflation variable: DerivedMetaData/elevation_diff_exp_inv # 2.0 + - filter: BlackList + where: + - variable: + name: MetaData/stationIdentification + is_in: {{ get_conventional_rejected_stations(observation_from_jcb) }} + actions: + - name: set + flag: rejectlist + ignore: rejected observations + - name: reject + obs post filters: + - filter: Background Check # gross error check + filter variables: + - name: totalSnowDepth + absolute threshold: 250 + bias correction parameter: 1.0 + actions: + - name: set + flag: background_check + ignore: rejected observations + - name: reject + diff --git a/parm/jcb-gdas/observations/snow/ims_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/ims_snow.yaml.j2 index 72e22eea2..a40980503 100644 --- a/parm/jcb-gdas/observations/snow/ims_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/ims_snow.yaml.j2 @@ -25,6 +25,20 @@ # Observation Filters (QC) # ------------------------ obs pre filters: + - filter: Create Diagnostic Flags + flags: + - name: land_check + initial value: false + - name: landice_check + initial value: false + - name: seaice_check + initial value: false + - name: invalid_snowdepth + initial value: false + - name: background_check + initial value: false + - name: spatial_thinning + initial value: false # assign observation error - filter: Perform Action filter variables: @@ -39,34 +53,63 @@ where: - variable: name: GeoVaLs/fraction_of_land - minvalue: 0.0001 + minvalue: 0.5 + actions: + - name: set + flag: land_check + ignore: rejected observations + - name: reject - filter: Domain Check # land only, no sea ice where: - variable: name: GeoVaLs/fraction_of_ice maxvalue: 0.0 + actions: + - name: set + flag: seaice_check + ignore: rejected observations + - name: reject - filter: RejectList # no land-ice where: - variable: name: GeoVaLs/vtype minvalue: 14.5 maxvalue: 15.5 + actions: + - name: set + flag: landice_check + ignore: rejected observations + - name: reject obs post filters: # gross error check - filter: Background Check filter variables: - name: totalSnowDepth - threshold: 3. - action: - name: reject + absolute threshold: 250 + actions: + - name: set + flag: background_check + ignore: rejected observations + - name: reject # thinning of remaining obs - filter: Gaussian Thinning horizontal_mesh: 40.0 # km + actions: + - name: set + flag: spatial_thinning + ignore: rejected observations + - name: reject # excludes where both obs & mod = 100% (set in IMS_proc) - filter: Bounds Check filter variables: - name: totalSnowDepth minvalue: 0.0 + actions: + - name: set + flag: invalid_snowdepth + ignore: rejected observations + - name: reject + diff --git a/parm/jcb-gdas/observations/snow/madis_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/madis_snow.yaml.j2 index 0f41bab1b..c388ffd91 100644 --- a/parm/jcb-gdas/observations/snow/madis_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/madis_snow.yaml.j2 @@ -21,11 +21,6 @@ # Observation Operator # -------------------- obs operator: - name: Composite - components: - - name: Identity - - name: BackgroundErrorIdentity - linear obs operator: name: Identity # @@ -48,28 +43,20 @@ initial value: false - name: landice_check initial value: false + - name: seaice_check + initial value: false - name: elevation_bkgdiff initial value: false - name: rejectlist initial value: false - name: background_check initial value: false - - name: buddy_check - initial value: false - filter: Perform Action filter variables: - name: totalSnowDepth action: name: assign error error parameter: 40.0 - - filter: Variable Assignment - assignments: - - name: GrossErrorProbability/totalSnowDepth - type: float - value: 0.02 - - name: BkgError/totalSnowDepth_background_error - type: float - value: 30.0 - filter: Domain Check where: - variable: @@ -115,7 +102,7 @@ where: - variable: name: GeoVaLs/fraction_of_land - minvalue: 0.0001 + minvalue: 0.5 actions: - name: set flag: land_check @@ -137,6 +124,11 @@ - variable: name: GeoVaLs/fraction_of_ice maxvalue: 0.0 + actions: + - name: set + flag: seaice_check + ignore: rejected observations + - name: reject - filter: RejectList # no land-ice where: - variable: @@ -214,7 +206,6 @@ - filter: Background Check # gross error check filter variables: - name: totalSnowDepth - threshold: 6.25 absolute threshold: 250 bias correction parameter: 1.0 actions: @@ -222,32 +213,4 @@ flag: background_check ignore: rejected observations - name: reject - - filter: Met Office Buddy Check - filter variables: - - name: totalSnowDepth - rejection_threshold: 0.5 - traced_boxes: # trace all observations - min_latitude: -90 - max_latitude: 90 - min_longitude: -180 - max_longitude: 180 - search_radius: 150 # km - station_id_variable: - name: MetaData/stationIdentification - num_zonal_bands: 24 - sort_by_pressure: false - max_total_num_buddies: 15 - max_num_buddies_from_single_band: 10 - max_num_buddies_with_same_station_id: 5 - use_legacy_buddy_collector: false - horizontal_correlation_scale: { "-90": 150, "90": 150 } - temporal_correlation_scale: PT6H - damping_factor_1: 1.0 - damping_factor_2: 1.0 - background_error_group: BkgError - actions: - - name: set - flag: buddy_check - ignore: rejected observations - - name: reject diff --git a/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 b/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 index 8b65b8a7a..df423d826 100644 --- a/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 @@ -11,7 +11,7 @@ mapping file: "{{snow_obsdatain_path}}/bufr_sfcsno_mapping.yaml" missing file action: warn obsgrouping: - group variables: [stationIdentification] + group variables: [stationIdentification] obsdataout: engine: type: H5File @@ -22,11 +22,6 @@ # Observation Operator # -------------------- obs operator: - name: Composite - components: - - name: Identity - - name: BackgroundErrorIdentity - linear obs operator: name: Identity # @@ -49,28 +44,20 @@ initial value: false - name: landice_check initial value: false + - name: seaice_check + initial value: false - name: elevation_bkgdiff initial value: false - name: rejectlist initial value: false - name: background_check initial value: false - - name: buddy_check - initial value: false - filter: Perform Action filter variables: - name: totalSnowDepth action: name: assign error error parameter: 40.0 - - filter: Variable Assignment - assignments: - - name: GrossErrorProbability/totalSnowDepth - type: float - value: 0.02 - - name: BkgError/totalSnowDepth_background_error - type: float - value: 30.0 - filter: Domain Check where: - variable: @@ -116,7 +103,7 @@ where: - variable: name: GeoVaLs/fraction_of_land - minvalue: 0.0001 + minvalue: 0.5 actions: - name: set flag: land_check @@ -138,6 +125,11 @@ - variable: name: GeoVaLs/fraction_of_ice maxvalue: 0.0 + actions: + - name: set + flag: seaice_check + ignore: rejected observations + - name: reject - filter: RejectList # no land-ice where: - variable: @@ -215,7 +207,6 @@ - filter: Background Check # gross error check filter variables: - name: totalSnowDepth - threshold: 6.25 absolute threshold: 250 bias correction parameter: 1.0 actions: @@ -223,32 +214,4 @@ flag: background_check ignore: rejected observations - name: reject - - filter: Met Office Buddy Check - filter variables: - - name: totalSnowDepth - rejection_threshold: 0.5 - traced_boxes: # trace all observations - min_latitude: -90 - max_latitude: 90 - min_longitude: -180 - max_longitude: 180 - search_radius: 150 # km - station_id_variable: - name: MetaData/stationIdentification - num_zonal_bands: 24 - sort_by_pressure: false - max_total_num_buddies: 15 - max_num_buddies_from_single_band: 10 - max_num_buddies_with_same_station_id: 5 - use_legacy_buddy_collector: false - horizontal_correlation_scale: { "-90": 150, "90": 150 } - temporal_correlation_scale: PT6H - damping_factor_1: 1.0 - damping_factor_2: 1.0 - background_error_group: BkgError - actions: - - name: set - flag: buddy_check - ignore: rejected observations - - name: reject diff --git a/parm/jcb-gdas/observations/snow/snocvr.yaml.j2 b/parm/jcb-gdas/observations/snow/snocvr.yaml.j2 index d5bc83050..f9b38a102 100644 --- a/parm/jcb-gdas/observations/snow/snocvr.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/snocvr.yaml.j2 @@ -22,11 +22,6 @@ # Observation Operator # -------------------- obs operator: - name: Composite - components: - - name: Identity - - name: BackgroundErrorIdentity - linear obs operator: name: Identity # @@ -49,28 +44,20 @@ initial value: false - name: landice_check initial value: false + - name: seaice_check + initial value: false - name: elevation_bkgdiff initial value: false - name: rejectlist initial value: false - name: background_check initial value: false - - name: buddy_check - initial value: false - filter: Perform Action filter variables: - name: totalSnowDepth action: name: assign error error parameter: 40.0 - - filter: Variable Assignment - assignments: - - name: GrossErrorProbability/totalSnowDepth - type: float - value: 0.02 - - name: BkgError/totalSnowDepth_background_error - type: float - value: 30.0 - filter: Domain Check where: - variable: @@ -116,7 +103,7 @@ where: - variable: name: GeoVaLs/fraction_of_land - minvalue: 0.0001 + minvalue: 0.5 actions: - name: set flag: land_check @@ -138,6 +125,11 @@ - variable: name: GeoVaLs/fraction_of_ice maxvalue: 0.0 + actions: + - name: set + flag: seaice_check + ignore: rejected observations + - name: reject - filter: RejectList # no land-ice where: - variable: @@ -215,7 +207,6 @@ - filter: Background Check # gross error check filter variables: - name: totalSnowDepth - threshold: 6.25 absolute threshold: 250 bias correction parameter: 1.0 actions: @@ -223,32 +214,4 @@ flag: background_check ignore: rejected observations - name: reject - - filter: Met Office Buddy Check - filter variables: - - name: totalSnowDepth - rejection_threshold: 0.5 - traced_boxes: # trace all observations - min_latitude: -90 - max_latitude: 90 - min_longitude: -180 - max_longitude: 180 - search_radius: 150 # km - station_id_variable: - name: MetaData/stationIdentification - num_zonal_bands: 24 - sort_by_pressure: false - max_total_num_buddies: 15 - max_num_buddies_from_single_band: 10 - max_num_buddies_with_same_station_id: 5 - use_legacy_buddy_collector: false - horizontal_correlation_scale: { "-90": 150, "90": 150 } - temporal_correlation_scale: PT6H - damping_factor_1: 1.0 - damping_factor_2: 1.0 - background_error_group: BkgError - actions: - - name: set - flag: buddy_check - ignore: rejected observations - - name: reject From c2e2fcc3cfbc37abbafbee261bc3aaf3179f3ba2 Mon Sep 17 00:00:00 2001 From: Anna Shlyaeva Date: Mon, 4 Aug 2025 07:38:43 -0600 Subject: [PATCH 140/199] Updates for faster recentering (#154) Faster recentering (short-term solution): - no extra copies of CICE restarts - only save recentering increment once from the jedi app: it's the same for all ensemble members - removed comments about ice recentering: this was done Co-authored-by: Anna Shlyaeva --- parm/jcb-gdas/algorithm/marine/soca_ens_handler.yaml.j2 | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/parm/jcb-gdas/algorithm/marine/soca_ens_handler.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_ens_handler.yaml.j2 index 99a112fe3..8c0652c18 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_ens_handler.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_ens_handler.yaml.j2 @@ -8,7 +8,6 @@ date: "{{ marine_window_end_iso }}" layers variable: [sea_water_cell_thickness] -# TODO(AFE) fix ice recentering in cycled da increment variables: - sea_water_potential_temperature - sea_water_salinity @@ -73,7 +72,7 @@ sea ice variable change: sno_lev: 1 cice output: pattern: "%mem%" - restart: ens/cice_model.res.output.%mem%.nc + restart: ens/cice_model.res.%mem%.nc output variables: - sea_water_potential_temperature - sea_water_salinity @@ -92,7 +91,6 @@ soca increments: # Could also be states, but they are read as increments read_from_file: 3 trajectory: -# TODO(AFE) fix ice recentering in cycled da state variables: - sea_water_potential_temperature - sea_water_salinity @@ -113,10 +111,8 @@ trajectory: output increment: datadir: ./ date: "{{ marine_window_end_iso }}" - exp: trash + exp: recenter type: incr - output file: "recenter.incr.%mem%.nc" - pattern: "%mem%" ensemble variance output: datadir: ./ From 725be4587013be8da632bc1b3be50ba02d5c001b Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Mon, 11 Aug 2025 16:46:15 -0400 Subject: [PATCH 141/199] Switch to using YAML based rescaling for aerosol B matrix (#159) Switch to using YAML based and not input file based rescaling for aerosol DA. Note that the rescaling is all 1.0 which could also be accomplished by omitting the whole section, but this is more explicit this way. --- .../aero/aero_gen_bmatrix_diagb.yaml.j2 | 49 ++++++------------- .../aero_gen_bmatrix_rescale_default.yaml.j2 | 37 ++++++++++++++ 2 files changed, 52 insertions(+), 34 deletions(-) create mode 100644 parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_rescale_default.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diagb.yaml.j2 b/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diagb.yaml.j2 index 05b1fb637..045991aed 100644 --- a/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diagb.yaml.j2 +++ b/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diagb.yaml.j2 @@ -9,6 +9,18 @@ geometry: npx: {{ aero_npx_anl }} npy: {{ aero_npy_anl }} npz: {{ aero_npz_anl }} + time invariant fields: + state fields: + datetime: '{{ aero_background_error_time_iso }}' + filetype: fms restart + skip coupler file: true + state variables: + - slmsk + field io names: + slmsk: slmsk + datapath: {{ aero_background_path }}/ + filename_orog: {{ aero_orog_prefix }}_oro_data.nc + filename_sfcd: {{ aero_background_error_time_fv3 }}.sfc_data.nc date: '{{ aero_background_error_time_iso }}' background: datetime: '{{ aero_background_error_time_iso }}' @@ -123,40 +135,9 @@ variables: - mass_fraction_of_sea_salt003_in_air - mass_fraction_of_sea_salt004_in_air -global rescale: - geometry: - fms initialization: - namelist filename: "{{aero_fv3jedi_files_path}}/fmsmpp.nml" - field table filename: "{{aero_fv3jedi_files_path}}/field_table" - akbk: "{{aero_fv3jedi_files_path}}/akbk.nc4" - layout: - - {{ aero_layout_x }} - - {{ aero_layout_y }} - npx: {{ aero_npx_clim_b }} - npy: {{ aero_npy_clim_b }} - npz: {{ aero_npz_clim_b }} - rescale stddev: - filetype: fms restart - skip coupler file: true - datapath: {{ aero_rescale_b_path }} - filename_trcr: rescale.fv_tracer.res.nc - filename_cplr: rescale.coupler.res - field io names: - mass_fraction_of_dust001_in_air: dust1 - mass_fraction_of_dust002_in_air: dust2 - mass_fraction_of_dust003_in_air: dust3 - mass_fraction_of_dust004_in_air: dust4 - mass_fraction_of_dust005_in_air: dust5 - mass_fraction_of_sea_salt001_in_air: seas1 - mass_fraction_of_sea_salt002_in_air: seas2 - mass_fraction_of_sea_salt003_in_air: seas3 - mass_fraction_of_sea_salt004_in_air: seas4 - mass_fraction_of_sea_salt005_in_air: seas5 - mass_fraction_of_hydrophobic_black_carbon_in_air: bc1 - mass_fraction_of_hydrophilic_black_carbon_in_air: bc2 - mass_fraction_of_hydrophobic_organic_carbon_in_air: oc1 - mass_fraction_of_hydrophilic_organic_carbon_in_air: oc2 - mass_fraction_of_sulfate_in_air: so4 +{% set rescaling_factor_file = aero_rescaling_factor_file|default(aero_gen_bmatrix_rescale_default) %} +{% include rescaling_factor_file %} + number of halo points: {{ aero_diagb_n_halo }} number of neighbors: {{ aero_diagb_n_neighbors }} simple smoothing: diff --git a/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_rescale_default.yaml.j2 b/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_rescale_default.yaml.j2 new file mode 100644 index 000000000..b5f80ed8a --- /dev/null +++ b/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_rescale_default.yaml.j2 @@ -0,0 +1,37 @@ +rescaling factors: + mass_fraction_of_sulfate_in_air: + rescaling factor: 1.0 + mass_fraction_of_hydrophobic_black_carbon_in_air: + ocean rescaling factor: 1.0 + land rescaling factor: 1.0 + mass_fraction_of_hydrophilic_black_carbon_in_air: + ocean rescaling factor: 1.0 + land rescaling factor: 1.0 + mass_fraction_of_hydrophobic_organic_carbon_in_air: + ocean rescaling factor: 1.0 + land rescaling factor: 1.0 + mass_fraction_of_hydrophilic_organic_carbon_in_air: + ocean rescaling factor: 1.0 + land rescaling factor: 1.0 + mass_fraction_of_sea_salt001_in_air: + ocean rescaling factor: 1.0 + land rescaling factor: 1.0 + mass_fraction_of_sea_salt002_in_air: + ocean rescaling factor: 1.0 + land rescaling factor: 1.0 + mass_fraction_of_sea_salt003_in_air: + ocean rescaling factor: 1.0 + land rescaling factor: 1.0 + mass_fraction_of_sea_salt004_in_air: + ocean rescaling factor: 1.0 + land rescaling factor: 1.0 + mass_fraction_of_dust001_in_air: + rescaling factor: 1.0 + mass_fraction_of_dust002_in_air: + rescaling factor: 1.0 + mass_fraction_of_dust003_in_air: + rescaling factor: 1.0 + mass_fraction_of_dust004_in_air: + rescaling factor: 1.0 + mass_fraction_of_dust005_in_air: + rescaling factor: 1.0 From 44c884ff8d81bf066fe474abebe00137433d03fb Mon Sep 17 00:00:00 2001 From: "Jianjun Jin, NOAA EMC" Date: Wed, 20 Aug 2025 15:16:43 -0400 Subject: [PATCH 142/199] Added configurations for AMSR2_GCOM-W1 (#133) Configurations to assimilate AMSR2_GCOM-W1 data. The observational errors are taken from those in GEOS. Comparison between JEDI and GSI results are posted in https://github.com/NOAA-EMC/GDASApp/issues/1536#issuecomment-2930644796 --------- Co-authored-by: Cory Martin --- .../atmosphere/amsr2_gcom-w1.yaml | 32 ++ .../atmosphere/amsr2_gcom-w1.yaml.j2 | 398 ++++++++++++++++++ 2 files changed, 430 insertions(+) create mode 100644 parm/jcb-gdas/observation_chronicle/atmosphere/amsr2_gcom-w1.yaml create mode 100644 parm/jcb-gdas/observations/atmosphere/amsr2_gcom-w1.yaml.j2 diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/amsr2_gcom-w1.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/amsr2_gcom-w1.yaml new file mode 100644 index 000000000..d445480a3 --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/atmosphere/amsr2_gcom-w1.yaml @@ -0,0 +1,32 @@ +# Instrument metadata +# ------------------- +commissioned: 2024-10-01T00:00:00 +observer_type: satellite # Type of chronicle to use + +# Instrument initial configuration +# -------------------------------- +channel_variables: + simulated: min + active: min + biascorrtd: min + error: max + error_cld: max + ermax: max + x0: max + x1: max +channel_values: +# sim act bcd err errcld ermax x0 x1 + 1: [ 1, -1, 1, 0.80, 5.00, 2.40, 0.050, 0.600] + 2: [ 1, -1, 1, 0.90, 5.00, 2.70, 0.100, 0.600] + 3: [ 1, -1, 1, 0.80, 5.00, 2.40, 0.050, 0.600] + 4: [ 1, -1, 1, 0.90, 5.00, 2.70, 0.050, 0.600] + 5: [ 1, -1, 1, 1.00, 5.00, 3.00, 0.100, 0.600] + 6: [ 1, -1, 1, 1.10, 18.50, 3.30, 0.100, 0.500] + 7: [ 1, 1, 1, 2.00, 20.00, 30.00, 0.050, 0.300] + 8: [ 1, 1, 1, 3.50, 40.00, 30.00, 0.050, 0.300] + 9: [ 1, 1, 1, 3.00, 20.00, 30.00, 0.050, 0.300] + 10: [ 1, 1, 1, 4.80, 25.00, 30.00, 0.050, 0.300] + 11: [ 1, 1, 1, 5.00, 30.00, 50.00, 0.050, 0.300] + 12: [ 1, 1, 1, 6.00, 30.00, 50.00, 0.050, 0.300] + 13: [ 1, 1, 1, 4.50, 30.00, 50.00, 0.050, 0.300] + 14: [ 1, -1, 1, 6.30, 20.00, 50.00, 0.050, 0.300] diff --git a/parm/jcb-gdas/observations/atmosphere/amsr2_gcom-w1.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/amsr2_gcom-w1.yaml.j2 new file mode 100644 index 000000000..31344ebff --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/amsr2_gcom-w1.yaml.j2 @@ -0,0 +1,398 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: AMSR2 GCOM-W1 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [brightnessTemperature] + channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} + + # Observation Operator + # -------------------- + obs operator: + name: CRTM + Absorbers: [H2O, O3, CO2] + Clouds: &{{observation_from_jcb}}_clouds [Water, Ice, Rain, Snow, Graupel] +# Cloud_Fraction: 1.0 +# ---methods for cloud fraction and radii in fov: thompson, or none + method for cloud fraction within fov: thompson + method for hydrometeor effective radii within fov: thompson + Cloud_Seeding: true + obs options: + Sensor_ID: &{{observation_from_jcb}}_sensor_id amsr2_gcom-w1 + EndianType: little_endian + CoefficientPath: "{{crtm_coefficient_path}}" + linear obs operator: + Absorbers: [H2O, O3] + + # Observation Bias Correction (VarBC) + # ----------------------------------- + obs bias: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" + variables without bc: [brightnessTemperature] + channels: {{ get_satellite_variable(observation_from_jcb, "not_biascorrtd") }} + variational bc: + predictors: + - name: constant + - name: lapseRate + order: 2 + tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" + - name: lapseRate + tlapse: *{{observation_from_jcb}}_tlapse + - name: emissivityJacobian + - name: sensorScanAngle + var_name: sensorScanPosition + order: 4 + - name: sensorScanAngle + var_name: sensorScanPosition + order: 3 + - name: sensorScanAngle + var_name: sensorScanPosition + order: 2 + - name: sensorScanAngle + var_name: sensorScanPosition + covariance: + minimal required obs number: 20 + variance range: [1.0e-6, 10.0] + step size: 1.0e-4 + largest analysis variance: 10000.0 + prior: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiascovin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovin_suffix}}" + inflation: + ratio: 1.1 + ratio for small dataset: 2.0 + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" + + # Observation Filters (QC) + # ------------------------ + obs pre filters: + # Data Thinning + - filter: Gaussian Thinning + horizontal_mesh: 100 + use_reduced_horizontal_grid: true + distance_norm: geodesic + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + action: + name: reduce obs space + + # ------------------------ + obs prior filters: + # Zero Atmospheric clouds in CRTM where water_area_fraction < 0.99 + - filter: Variable Assignment + assignments: + - name: MetaData/zeroCloudInCRTM + type: int + function: + name: ObsFunction/Conditional + options: + # firstmatchingcase: false + defaultvalue: 0 # Will not zero clouds by default + cases: + - where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + value: 1 # Will zero clouds by default + + obs post filters: + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + minvalue: 50.0 + maxvalue: 340.0 + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: GeoVaLs/water_area_fraction + minvalue: 0.99 + - variable: + name: GeoVaLs/skin_temperature_at_surface_where_sea + minvalue: 275 + - variable: + name: GeoVaLs/wind_speed_at_surface + maxvalue: 12 + - variable: + name: MetaData/latitude + minvalue: -60.0 + maxvalue: 60.0 + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/TotalColumnVaporGuess + minvalue: 10.0 + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/SunGlintAngle + minvalue: 20.0 + + # Save CLW retrievals from ObsValue + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/clw_obs + type: float + function: + name: ObsFunction/CLWRetMW + options: + clwret_ch18v: 7 + clwret_ch18h: 8 + clwret_ch36v: 11 + clwret_ch36h: 12 + sys_bias: [0.4800, 3.0737, 0.7433, 3.6430, + 3.5304, 4.4270, 5.1448, 5.0785, + 4.9763, 9.3215, 2.5789, 5.5274, + 0.6641, 1.3674] + clwret_types: [ObsValue] + # Save CLW retrievals from HofX + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/clw_guess_retrieval + type: float + function: + name: ObsFunction/CLWRetMW + options: + clwret_ch18v: 7 + clwret_ch18h: 8 + clwret_ch36v: 11 + clwret_ch36h: 12 + sys_bias: [0.4800, 3.0737, 0.7433, 3.6430, + 3.5304, 4.4270, 5.1448, 5.0785, + 4.9763, 9.3215, 2.5789, 5.5274, + 0.6641, 1.3674] + clwret_types: [HofX] + # Ckeck CLW retrievals from observations + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: DerivedMetaData/clw_obs + maxvalue: 1.0 + # Ckeck CLW retrievals from HofX + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: DerivedMetaData/clw_guess_retrieval + maxvalue: 1.0 + - filter: Difference Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + value: + name: DerivedMetaData/clw_obs + reference: + name: DerivedMetaData/clw_guess_retrieval + minvalue: -0.5 + maxvalue: 0.5 + # Combined (clw_obs + clw_guess_retrieval)/2.0 + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/CLWRetSymmetricMW + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: DerivedMetaData/clw_obs + - name: DerivedMetaData/clw_guess_retrieval + coefs: [0.5, 0.5] + - filter: BlackList + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelRamp + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + xvar: + name: DerivedMetaData/CLWRetSymmetricMW + x0: &{{observation_from_jcb}}_x0 [{{ get_satellite_variable(observation_from_jcb, "x0") }}] + x1: &{{observation_from_jcb}}_x1 [{{ get_satellite_variable(observation_from_jcb, "x1") }}] + err0: &{{observation_from_jcb}}_err0 [{{ get_satellite_variable(observation_from_jcb, "error") }}] + err1: &{{observation_from_jcb}}_err1 [{{ get_satellite_variable(observation_from_jcb, "error_cld") }}] + + ## Prepare variables used by cold-air-outbreak checks in all-sky. + # Combined ColumnLiquidCloud + ColumnIceCloud + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnLiquidandIceCloud + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: GeoVaLs/mass_content_of_cloud_liquid_water_in_atmosphere_column + - name: GeoVaLs/mass_content_of_cloud_ice_in_atmosphere_column + coefs: [1, 1] + # replace zero by 9.99e-7 in DerivedMetaData/ColumnLiquidandIceCloud + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnLiquidandIceCloud + type: float + value: 9.99e-07 + where: + - variable: + name: DerivedMetaData/ColumnLiquidandIceCloud + maxvalue: 9.99e-07 + + # ratio of liquid /(Liquid+Ice) + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnRatioLiquidCloud + type: float + function: + name: ObsFunction/ElementMultiply + options: + variables: + - name: GeoVaLs/mass_content_of_cloud_liquid_water_in_atmosphere_column + - name: DerivedMetaData/ColumnLiquidandIceCloud + exponents: [1,-1] + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnRatioLiquidCloud + type: float + value: 0.0 + where: + - variable: + name: DerivedMetaData/ColumnLiquidandIceCloud + maxvalue: 1.0e-06 + + # TotalColumnVaporGuess + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/TotalColumnVaporGuess + type: float + function: + name: ObsFunction/TotalColumnVaporGuess + + # potentialTemperature at surface + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureSurface + type: float + function: + name: PotentialTemperatureFromTemperature@ObsFunction + options: + use surface pressure: true + # potentialTemperature near 700 hPa + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureAt700hPa + type: float + function: + name: PotentialTemperatureFromTemperature@ObsFunction + options: + use surface pressure: false + pressure to evaluate potential temperature: 70000.0 + # stability = Difference between PotentialTemperatur values + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureDiff + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: DerivedMetaData/PotentialTemperatureAt700hPa + - name: DerivedMetaData/PotentialTemperatureSurface + coefs: [1, -1] + # cold-air-outbreak (cao) check for all-sky DA with Rain, Snow, or Graupel + - filter: BlackList + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: ObsFunction/TotalColumnVaporGuess + maxvalue: 8.0 + - variable: + name: DerivedMetaData/ColumnRatioLiquidCloud + maxvalue: 0.5 + - variable: + name: DerivedMetaData/PotentialTemperatureDiff + maxvalue: 12.0 + - variable: + name: GeoVaLs/water_area_fraction + minvalue: 0.99 + action: + name: reject + + # Background check + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: 1-6 + threshold: 3.0 + action: + name: reject + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: 7-14 + threshold: 2.0 + action: + name: reject + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: 7-10 + absolute threshold: 30 + action: + name: reject + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: 11-14 + absolute threshold: 50 + action: + name: reject + # Useflag check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/ChannelUseflagCheckRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + # use passive_bc: true + sensor: *{{observation_from_jcb}}_sensor_id + use_flag: &{{observation_from_jcb}}_active_channels [{{ get_satellite_variable(observation_from_jcb, "active") }}] + minvalue: 1.0e-12 + action: + name: reject + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 From e7617040c46203646270eb531bcb1a1108000855 Mon Sep 17 00:00:00 2001 From: "Jianjun Jin, NOAA EMC" Date: Mon, 25 Aug 2025 08:42:53 -0400 Subject: [PATCH 143/199] Activated the Thompson method calculate cloud fraction and particle radii (#118) Activated the calculations of cloud fraction and hydrometeor effective radii by the Thompson method in assimilation of AMSUA data in all-sky conditions. It requres the updates of ctests' references in https://github.com/NOAA-EMC/GDASApp/pull/1673 # Issues Resolved the issue https://github.com/NOAA-EMC/GDASApp/issues/1753 Co-authored-by: Dan Holdaway <27729500+danholdaway@users.noreply.github.com> Co-authored-by: Cory Martin --- .../observations/atmosphere/amsua_n19.yaml.j2 | 30 +++++++++++++++++-- 1 file changed, 27 insertions(+), 3 deletions(-) diff --git a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 index 319eed08d..d10346d8f 100644 --- a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 @@ -23,10 +23,10 @@ name: CRTM Absorbers: [H2O, O3, CO2] Clouds: &{{observation_from_jcb}}_clouds [Water, Ice, Rain, Snow, Graupel] - Cloud_Fraction: 1.0 +# Cloud_Fraction: 1.0 # ---methods for cloud fraction and radii in fov: thompson, or none - method for cloud fraction within fov: none - method for hydrometeor effective radii within fov: none + method for cloud fraction within fov: thompson + method for hydrometeor effective radii within fov: thompson Cloud_Seeding: true obs options: Sensor_ID: &{{observation_from_jcb}}_sensor_id amsua_n19 @@ -71,6 +71,30 @@ output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" # Observation Filters (QC) + # ------------------------ + obs pre filters: + # Remove Observations from the Edge of the Scan + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: 1-15 + where: + - variable: + name: MetaData/sensorScanPosition + is_in: 4-27 + action: + name: reduce obs space + + # Data Thinning + - filter: Gaussian Thinning + horizontal_mesh: 145 + use_reduced_horizontal_grid: true + distance_norm: geodesic + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + action: + name: reduce obs space + # ------------------------ obs prior filters: # Zero Atmospheric clouds in CRTM where water_area_fraction < 0.99 From 203dc2ab076f54117f1c6b74e1774ab4936482c9 Mon Sep 17 00:00:00 2001 From: Anna Shlyaeva Date: Wed, 27 Aug 2025 09:56:18 -0600 Subject: [PATCH 144/199] Configuration changes for parallel marine ensemble recenter (#160) Keeping as draft until https://github.com/JCSDA-internal/oops/pull/2970 and https://github.com/JCSDA-internal/soca/pull/1183 are merged. --------- Co-authored-by: Anna Shlyaeva --- .../algorithm/marine/soca_ens_handler.yaml.j2 | 147 ++++++++---------- 1 file changed, 68 insertions(+), 79 deletions(-) diff --git a/parm/jcb-gdas/algorithm/marine/soca_ens_handler.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_ens_handler.yaml.j2 index 8c0652c18..661e1492d 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_ens_handler.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_ens_handler.yaml.j2 @@ -1,14 +1,12 @@ # This yaml is for applying deterministic recentering increments to the ensemble members - geometry: mom6_input_nml: mom_input.nml fields metadata: ./fields_metadata.yaml -date: "{{ marine_window_end_iso }}" - -layers variable: [sea_water_cell_thickness] +nens: '{{ marine_number_ensemble_members }}' +nens per MPI task: 2 -increment variables: +increment variables: &vars - sea_water_potential_temperature - sea_water_salinity - eastward_sea_water_velocity @@ -16,81 +14,28 @@ increment variables: - sea_ice_area_fraction - sea_ice_thickness - sea_ice_snow_thickness -set increment variables to zero: -- eastward_sea_water_velocity -- northward_sea_water_velocity -vertical geometry: - date: "{{ marine_window_begin_iso }}" - basename: ./INPUT/ - ocn_filename: MOM.res.nc - read_from_file: 3 - -add recentering increment: true -recentering around deterministic: true -sea ice recenter: true -sea ice analysis: - pattern: "%mem%" - date: "{{ marine_window_end_iso }}" - ocn_filename: "ocean.%mem%.nc" - ice_filename: "ice.%mem%.nc" - read_from_file: 1 - basename: {{ enspert_relpath }}/ens/ - state variables: - - sea_water_potential_temperature - - sea_water_salinity - - eastward_sea_water_velocity - - northward_sea_water_velocity - - sea_water_cell_thickness - - sea_water_depth - - ocean_mixed_layer_thickness - - sea_ice_area_fraction - - sea_ice_thickness - - sea_ice_snow_thickness - -sea ice variable change: - variable change name: Soca2Cice - arctic: - seaice edge: 0.4 - shuffle: true - rescale prior: - rescale: true - min hice: 0.5 - min hsno: 0.1 - antarctic: - seaice edge: 0.4 - shuffle: true - rescale prior: - rescale: true - min hice: 0.5 - min hsno: 0.1 - cice background state: +backgrounds: + members from template: + nmembers: '{{ marine_number_ensemble_members }}' pattern: "%mem%" - restart: ens/cice_model.res.%mem%.nc - ncat: 5 - ice_lev: 7 - sno_lev: 1 - cice output: - pattern: "%mem%" - restart: ens/cice_model.res.%mem%.nc - output variables: - - sea_water_potential_temperature - - sea_water_salinity - - sea_ice_area_fraction - - sea_ice_thickness - - sea_ice_snow_thickness + template: + date: "{{ marine_window_end_iso }}" + basename: ./ens/ + ocn_filename: "ocean.%mem%.nc" + ice_filename: "ice.%mem%.nc" + read_from_file: 1 + state variables: + - sea_water_potential_temperature + - sea_water_salinity + - sea_water_cell_thickness + - eastward_sea_water_velocity + - northward_sea_water_velocity + - sea_ice_area_fraction + - sea_ice_thickness + - sea_ice_snow_thickness -soca increments: # Could also be states, but they are read as increments - number of increments: '{{ marine_number_ensemble_members }}' - pattern: "%mem%" - template: - date: "{{ marine_window_end_iso }}" - basename: ./ens/ - ocn_filename: "ocean.%mem%.nc" - ice_filename: "ice.%mem%.nc" - read_from_file: 3 - -trajectory: +recentering state: state variables: - sea_water_potential_temperature - sea_water_salinity @@ -108,11 +53,55 @@ trajectory: ice_filename: ice.bkg.f009.nc read_from_file: 1 -output increment: +output increments: datadir: ./ date: "{{ marine_window_end_iso }}" exp: recenter - type: incr + type: ens + +increment postprocessing: + append vertical geometry: + layers variable: sea_water_cell_thickness + vertical geometry: + date: "{{ marine_window_begin_iso }}" + basename: ./INPUT/ + ocn_filename: MOM.res.nc + read_from_file: 3 + set increment variables to zero: + - eastward_sea_water_velocity + - northward_sea_water_velocity + +analysis postprocessing: + sea ice variable change: + variable change name: Soca2Cice + pattern: "%mem%" + arctic: + seaice edge: 0.4 + shuffle: true + rescale prior: + rescale: true + min hice: 0.5 + min hsno: 0.1 + antarctic: + seaice edge: 0.4 + shuffle: true + rescale prior: + rescale: true + min hice: 0.5 + min hsno: 0.1 + cice background state: + restart: ens/cice_model.res.%mem%.nc + ncat: 5 + ice_lev: 7 + sno_lev: 1 + cice output: + restart: ens/cice_model.res.%mem%.nc + output variables: + - sea_water_potential_temperature + - sea_water_salinity + - sea_ice_area_fraction + - sea_ice_thickness + - sea_ice_snow_thickness ensemble variance output: datadir: ./ From 146baff80ff997073997fe09fa0057979a430236 Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Tue, 2 Sep 2025 08:23:48 -0400 Subject: [PATCH 145/199] Template for snow DA SCF preprocessor code (#161) Add in a template for a new C++ OOPS application that preprocesses snow cover fraction observations (like IMS) using the model background to produce IODA observations for assimilation --- .../snow/snow_ims_scf_preprocess.yaml.j2 | 40 +++++++++++++++++++ .../model/snow/snow_background.yaml.j2 | 1 + 2 files changed, 41 insertions(+) create mode 100644 parm/jcb-gdas/algorithm/snow/snow_ims_scf_preprocess.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/snow/snow_ims_scf_preprocess.yaml.j2 b/parm/jcb-gdas/algorithm/snow/snow_ims_scf_preprocess.yaml.j2 new file mode 100644 index 000000000..987234355 --- /dev/null +++ b/parm/jcb-gdas/algorithm/snow/snow_ims_scf_preprocess.yaml.j2 @@ -0,0 +1,40 @@ +geometry: + fms initialization: + namelist filename: ./fv3jedi/fmsmpp.nml + field table filename: ./fv3jedi/field_table + akbk: ./fv3jedi/akbk.nc4 + layout: [1, 1] + npx: {{ snow_npx_ges }} + npy: {{ snow_npy_ges }} + npz: {{ snow_npz_ges }} + time invariant fields: + state fields: + datapath: {{ snow_background_path }} + filetype: fms restart + skip coupler file: true + datetime: '{{ snow_background_time_iso }}' + state variables: + - filtered_orography + field io names: + filtered_orography: orog_filt + filename_sfcd: '{{ snow_background_time_fv3 }}.sfc_data.nc' + filename_cplr: '{{ snow_background_time_fv3 }}.coupler.res' + filename_orog: '{{ snow_orog_prefix }}_oro_data.nc' +date: '{{ snow_background_time_iso }}' +output ioda file: '{{snow_obsdatain_path}}/{{snow_obsdatain_prefix}}ims_snow.tm00.nc' +input scf file: './obs/ims{{ snow_background_time_julian }}_4km_v1.3.asc' +mapping file: './obs/IMS4km_to_FV3_mapping.{{ snow_orog_prefix }}_oro_data.nc' +variables: +- totalSnowDepth +- vtype +- slmsk +- sheleg +- filtered_orography +- fraction_of_ice +- fraction_of_land +- stc +background: +{% filter indent(width=2) %} +{% set background_file = background_file|default(model_component ~ 'background', true) ~ '.yaml.j2' %} +{% include background_file %} +{% endfilter %} diff --git a/parm/jcb-gdas/model/snow/snow_background.yaml.j2 b/parm/jcb-gdas/model/snow/snow_background.yaml.j2 index 2c718220b..1f1ccb57b 100644 --- a/parm/jcb-gdas/model/snow/snow_background.yaml.j2 +++ b/parm/jcb-gdas/model/snow/snow_background.yaml.j2 @@ -10,6 +10,7 @@ state variables: - filtered_orography - fraction_of_ice - fraction_of_land +- stc field io names: totalSnowDepth: snodl filtered_orography: orog_filt From 484838a888eb6e4417de276fb0d7c0cb9a3a84ae Mon Sep 17 00:00:00 2001 From: JohnSteffen-NOAA <77806417+JohnSteffen-NOAA@users.noreply.github.com> Date: Tue, 2 Sep 2025 16:28:07 -0400 Subject: [PATCH 146/199] Updates to marine DA QC yamls for ObsForge (#165) This PR incorporates changes to the marine DA QC yamls to work with ObsForge data processing for real-time experiments. Here is a detailed list of the changes: 1. ADTs are split into separate obs spaces and the error coefficient is adjusted. 2. A background check QC is applied to AMSR2 sea ice retrievals. See [https://github.com/NOAA-EMC/GDASApp/issues/1868](url) 3. A QC yaml file is generated for surface drifters to match naming convention from ObsForge 4. QC yamls for some SST obs spaces are changed to match naming convention from ObsForge Co-authored-by: john steffen Co-authored-by: Guillaume Vernieres --- .../observations/marine/adt_rads_all.yaml.j2 | 2 +- .../marine/icec_amsr2_north.yaml.j2 | 4 +- .../marine/icec_amsr2_south.yaml.j2 | 4 +- .../insitu_temp_surface_drifter.yaml.j2 | 60 +++++++++++++++++++ .../observations/marine/rads_adt_3a.yaml.j2 | 1 + .../observations/marine/rads_adt_3b.yaml.j2 | 1 + .../observations/marine/rads_adt_6a.yaml.j2 | 1 + .../observations/marine/rads_adt_c2.yaml.j2 | 1 + .../observations/marine/rads_adt_j2.yaml.j2 | 1 + .../observations/marine/rads_adt_j3.yaml.j2 | 1 + .../observations/marine/rads_adt_sa.yaml.j2 | 1 + .../observations/marine/rads_adt_sw.yaml.j2 | 1 + .../marine/sst_avhrrf_ma_l3u.yaml.j2 | 1 + .../marine/sst_avhrrf_mb_l3u.yaml.j2 | 1 + .../marine/sst_avhrrf_mc_l3u.yaml.j2 | 1 + .../marine/sst_viirs_n21_l3u.yaml.j2 | 1 + 16 files changed, 77 insertions(+), 5 deletions(-) create mode 100644 parm/jcb-gdas/observations/marine/insitu_temp_surface_drifter.yaml.j2 create mode 120000 parm/jcb-gdas/observations/marine/rads_adt_3a.yaml.j2 create mode 120000 parm/jcb-gdas/observations/marine/rads_adt_3b.yaml.j2 create mode 120000 parm/jcb-gdas/observations/marine/rads_adt_6a.yaml.j2 create mode 120000 parm/jcb-gdas/observations/marine/rads_adt_c2.yaml.j2 create mode 120000 parm/jcb-gdas/observations/marine/rads_adt_j2.yaml.j2 create mode 120000 parm/jcb-gdas/observations/marine/rads_adt_j3.yaml.j2 create mode 120000 parm/jcb-gdas/observations/marine/rads_adt_sa.yaml.j2 create mode 120000 parm/jcb-gdas/observations/marine/rads_adt_sw.yaml.j2 create mode 120000 parm/jcb-gdas/observations/marine/sst_avhrrf_ma_l3u.yaml.j2 create mode 120000 parm/jcb-gdas/observations/marine/sst_avhrrf_mb_l3u.yaml.j2 create mode 120000 parm/jcb-gdas/observations/marine/sst_avhrrf_mc_l3u.yaml.j2 create mode 120000 parm/jcb-gdas/observations/marine/sst_viirs_n21_l3u.yaml.j2 diff --git a/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 b/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 index d9441cf37..c1db68548 100644 --- a/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 @@ -42,7 +42,7 @@ variables: [GeoVaLs/mesoscale_representation_error, ObsError/absoluteDynamicTopography] coefs: [0.2, - 1.0] + 3.0] - filter: Domain Check where: - variable: { name: GeoVaLs/sea_ice_area_fraction} diff --git a/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 index 1dd133033..06d647c42 100644 --- a/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 @@ -27,8 +27,8 @@ - filter: Bounds Check minvalue: 0.0 maxvalue: 1.0 - #- filter: Background Check - # threshold: 5.0 + - filter: Background Check + absolute threshold: 0.5 - filter: Domain Check where: - variable: { name: GeoVaLs/sea_surface_temperature} diff --git a/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 index 1dd133033..06d647c42 100644 --- a/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 @@ -27,8 +27,8 @@ - filter: Bounds Check minvalue: 0.0 maxvalue: 1.0 - #- filter: Background Check - # threshold: 5.0 + - filter: Background Check + absolute threshold: 0.5 - filter: Domain Check where: - variable: { name: GeoVaLs/sea_surface_temperature} diff --git a/parm/jcb-gdas/observations/marine/insitu_temp_surface_drifter.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_temp_surface_drifter.yaml.j2 new file mode 100644 index 000000000..87a15170c --- /dev/null +++ b/parm/jcb-gdas/observations/marine/insitu_temp_surface_drifter.yaml.j2 @@ -0,0 +1,60 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaSurfaceTemperature] + observed variables: [seaSurfaceTemperature] + io pool: + max pool size: 1 + get values: + time interpolation: linear + obs operator: + name: Identity + observation alias file: obsop_name_map.yaml + obs error: + covariance model: diagonal + + obs pre filters: + - filter: Bounds Check + minvalue: -1.2 + maxvalue: 41.0 + - filter: Domain Check + where: + - variable: {name: ObsError/seaSurfaceTemperature} + minvalue: 1.0e-8 + + obs prior filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + value: is_valid + minvalue: 0.9 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_ice_area_fraction} + maxvalue: 1.0e-5 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_surface_temperature} + minvalue: -1.2 + + obs post filters: + - filter: Background Check + absolute threshold: 5.0 + +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} + diff --git a/parm/jcb-gdas/observations/marine/rads_adt_3a.yaml.j2 b/parm/jcb-gdas/observations/marine/rads_adt_3a.yaml.j2 new file mode 120000 index 000000000..02ad0a9d7 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/rads_adt_3a.yaml.j2 @@ -0,0 +1 @@ +adt_rads_all.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/rads_adt_3b.yaml.j2 b/parm/jcb-gdas/observations/marine/rads_adt_3b.yaml.j2 new file mode 120000 index 000000000..02ad0a9d7 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/rads_adt_3b.yaml.j2 @@ -0,0 +1 @@ +adt_rads_all.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/rads_adt_6a.yaml.j2 b/parm/jcb-gdas/observations/marine/rads_adt_6a.yaml.j2 new file mode 120000 index 000000000..02ad0a9d7 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/rads_adt_6a.yaml.j2 @@ -0,0 +1 @@ +adt_rads_all.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/rads_adt_c2.yaml.j2 b/parm/jcb-gdas/observations/marine/rads_adt_c2.yaml.j2 new file mode 120000 index 000000000..02ad0a9d7 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/rads_adt_c2.yaml.j2 @@ -0,0 +1 @@ +adt_rads_all.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/rads_adt_j2.yaml.j2 b/parm/jcb-gdas/observations/marine/rads_adt_j2.yaml.j2 new file mode 120000 index 000000000..02ad0a9d7 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/rads_adt_j2.yaml.j2 @@ -0,0 +1 @@ +adt_rads_all.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/rads_adt_j3.yaml.j2 b/parm/jcb-gdas/observations/marine/rads_adt_j3.yaml.j2 new file mode 120000 index 000000000..02ad0a9d7 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/rads_adt_j3.yaml.j2 @@ -0,0 +1 @@ +adt_rads_all.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/rads_adt_sa.yaml.j2 b/parm/jcb-gdas/observations/marine/rads_adt_sa.yaml.j2 new file mode 120000 index 000000000..02ad0a9d7 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/rads_adt_sa.yaml.j2 @@ -0,0 +1 @@ +adt_rads_all.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/rads_adt_sw.yaml.j2 b/parm/jcb-gdas/observations/marine/rads_adt_sw.yaml.j2 new file mode 120000 index 000000000..02ad0a9d7 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/rads_adt_sw.yaml.j2 @@ -0,0 +1 @@ +adt_rads_all.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/sst_avhrrf_ma_l3u.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_avhrrf_ma_l3u.yaml.j2 new file mode 120000 index 000000000..5298b9a81 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/sst_avhrrf_ma_l3u.yaml.j2 @@ -0,0 +1 @@ +sst_generic.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/sst_avhrrf_mb_l3u.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_avhrrf_mb_l3u.yaml.j2 new file mode 120000 index 000000000..5298b9a81 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/sst_avhrrf_mb_l3u.yaml.j2 @@ -0,0 +1 @@ +sst_generic.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/sst_avhrrf_mc_l3u.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_avhrrf_mc_l3u.yaml.j2 new file mode 120000 index 000000000..5298b9a81 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/sst_avhrrf_mc_l3u.yaml.j2 @@ -0,0 +1 @@ +sst_generic.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/sst_viirs_n21_l3u.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_viirs_n21_l3u.yaml.j2 new file mode 120000 index 000000000..5298b9a81 --- /dev/null +++ b/parm/jcb-gdas/observations/marine/sst_viirs_n21_l3u.yaml.j2 @@ -0,0 +1 @@ +sst_generic.yaml.j2 \ No newline at end of file From 966d40ccc6d1a73e2f216f130c90e484027590bc Mon Sep 17 00:00:00 2001 From: "Jianjun Jin, NOAA EMC" Date: Mon, 8 Sep 2025 09:03:53 -0400 Subject: [PATCH 147/199] Activated the Thompson method for ATMS NPP, N20 and N21 (#163) Activated the Thompson method for ATMS; set "round_horizontal_bin_count_to_nearest: true". Co-authored-by: Cory Martin --- parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 | 8 ++++---- parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 | 8 ++++---- parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 | 8 ++++---- 3 files changed, 12 insertions(+), 12 deletions(-) diff --git a/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 index 6cbfbac7b..82e048949 100644 --- a/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 @@ -23,10 +23,10 @@ name: CRTM Absorbers: [H2O, O3, CO2] Clouds: &{{observation_from_jcb}}_clouds [Water, Ice, Rain, Snow, Graupel] - Cloud_Fraction: 1.0 +# Cloud_Fraction: 1.0 # ---methods for cloud fraction and radii in fov: thompson, or none - method for cloud fraction within fov: none - method for hydrometeor effective radii within fov: none + method for cloud fraction within fov: thompson + method for hydrometeor effective radii within fov: thompson Cloud_Seeding: true obs options: Sensor_ID: &{{observation_from_jcb}}_sensor_id atms_n20 @@ -89,7 +89,7 @@ horizontal_mesh: 145 use_reduced_horizontal_grid: true distance_norm: geodesic - # round_horizontal_bin_count_to_nearest: true + round_horizontal_bin_count_to_nearest: true # partition_longitude_bins_using_mesh: true action: name: reduce obs space diff --git a/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 index 83b627baa..63da3fa52 100644 --- a/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 @@ -23,10 +23,10 @@ name: CRTM Absorbers: [H2O, O3, CO2] Clouds: &{{observation_from_jcb}}_clouds [Water, Ice, Rain, Snow, Graupel] - Cloud_Fraction: 1.0 +# Cloud_Fraction: 1.0 # ---methods for cloud fraction and radii in fov: thompson, or none - method for cloud fraction within fov: none - method for hydrometeor effective radii within fov: none + method for cloud fraction within fov: thompson + method for hydrometeor effective radii within fov: thompson Cloud_Seeding: true obs options: Sensor_ID: &{{observation_from_jcb}}_sensor_id atms_n21 @@ -89,7 +89,7 @@ horizontal_mesh: 145 use_reduced_horizontal_grid: true distance_norm: geodesic - # round_horizontal_bin_count_to_nearest: true + round_horizontal_bin_count_to_nearest: true # partition_longitude_bins_using_mesh: true action: name: reduce obs space diff --git a/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 index f60300209..cdb0e4ef2 100644 --- a/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 @@ -23,10 +23,10 @@ name: CRTM Absorbers: [H2O, O3, CO2] Clouds: &{{observation_from_jcb}}_clouds [Water, Ice, Rain, Snow, Graupel] - Cloud_Fraction: 1.0 +# Cloud_Fraction: 1.0 # ---methods for cloud fraction and radii in fov: thompson, or none - method for cloud fraction within fov: none - method for hydrometeor effective radii within fov: none + method for cloud fraction within fov: thompson + method for hydrometeor effective radii within fov: thompson Cloud_Seeding: true obs options: Sensor_ID: &{{observation_from_jcb}}_sensor_id atms_npp @@ -89,7 +89,7 @@ horizontal_mesh: 145 use_reduced_horizontal_grid: true distance_norm: geodesic - # round_horizontal_bin_count_to_nearest: true + round_horizontal_bin_count_to_nearest: true # partition_longitude_bins_using_mesh: true action: name: reduce obs space From d67bb8a196d6be49c8baba849192038dbb28818d Mon Sep 17 00:00:00 2001 From: emilyhcliu <36091766+emilyhcliu@users.noreply.github.com> Date: Mon, 8 Sep 2025 09:16:53 -0400 Subject: [PATCH 148/199] Update obs yamls and obs chronicale for various observation types (#167) This PR updates the following: 1. Rename the filename of the observation YAMLs for scatwind and ozone so they are consistent with their ioda filenames 2. Update variable names so they follow JCSDA IODA and model variable naming convention. 3. Update data thinning and add reduce obs space 4. Update observation YAML files for CrIS 5. Add observation YAMLs for new satellite platforms 6. Add new platforms in the atmosphere chronicle For the observation types listed below, updates in this PR have been tested in global-workflow using GDASApp 3dvar test configuration for consistency with bufr2ioda processing and observer performance (HofX, Thinning, and Filtering): ``` - conventional_ps - scatwnd.ascat_metop-b - scatwnd.ascat_metop-c - satwnd.abi_goes-16 - satwnd.abi_goes-18 - satwnd.leogeo_multi - satwnd.viirs_n20 - satwnd.viirs_npp - satwnd.seviri_m9 - satwnd.seviri_m10 - satwnd.ahi_h9 - satwnd.avhrr_n15 - satwnd.avhrr_n18 - satwnd.avhrr_n19 - ozone.omi_aura - ozone.ompsnp_n20 - ozone.ompsnp_npp - ozone.ompstc_n20 - ozone.ompstc_npp - iasi_metop-b - iasi_metop-c - cris-fsr_n20 - cris-fsr_n21 - atms_npp - atms_n20 - gnssro_cosmic2 - gnssro_spire - gnssro_s6 - gnssro_geoopt - gnssro_grace - gnssro_k5 - gnssro_metop - gnssro_paz - gnssro_piq - gnssro_tsx - gnssro_tdm ``` Notes: 1. ATMS YAML is not updated in this PR. It was copied from [jcb-gdas PR #163 ](https://github.com/NOAA-EMC/jcb-gdas/pull/163) to test and validate. 2. The [GDASApp PR #1883](https://github.com/NOAA-EMC/GDASApp/pull/1883) is relevant to this PR. --------- Co-authored-by: Cory Martin --- .../atmosphere/cris-fsr_n21.yaml | 451 ++++++++++++ .../atmosphere/iasi_metop-c.yaml | 637 ++++++++++++++++ .../atmosphere/conventional_ps.yaml.j2 | 5 + .../atmosphere/cris-fsr_n20.yaml.j2 | 686 ++++++++---------- .../atmosphere/cris-fsr_n21.yaml.j2 | 376 ++++++++++ .../atmosphere/cris-fsr_npp.yaml.j2 | 685 ++++++++--------- .../atmosphere/iasi_metop-a.yaml.j2 | 44 +- .../atmosphere/iasi_metop-b.yaml.j2 | 44 +- .../atmosphere/iasi_metop-c.yaml.j2 | 317 ++++++++ ...mi_aura.yaml.j2 => ozone.omi_aura.yaml.j2} | 69 +- .../atmosphere/ozone.ompsnp_n20.yaml.j2 | 332 +++++++++ ...p_npp.yaml.j2 => ozone.ompsnp_npp.yaml.j2} | 10 +- .../atmosphere/ozone.ompstc_n20.yaml.j2 | 133 ++++ ...c_npp.yaml.j2 => ozone.ompstc_npp.yaml.j2} | 6 +- .../atmosphere/satwnd.abi_goes-16.yaml.j2 | 5 + .../atmosphere/satwnd.abi_goes-17.yaml.j2 | 7 +- .../atmosphere/satwnd.abi_goes-18.yaml.j2 | 7 +- .../atmosphere/satwnd.ahi_h8.yaml.j2 | 136 ++-- .../atmosphere/satwnd.ahi_h9.yaml.j2 | 136 ++-- .../atmosphere/satwnd.avhrr_metop-a.yaml.j2 | 289 ++++++++ .../atmosphere/satwnd.avhrr_metop-b.yaml.j2 | 289 ++++++++ .../atmosphere/satwnd.avhrr_metop-c.yaml.j2 | 289 ++++++++ .../atmosphere/satwnd.avhrr_n15.yaml.j2 | 2 +- .../atmosphere/satwnd.avhrr_n18.yaml.j2 | 2 +- .../atmosphere/satwnd.avhrr_n19.yaml.j2 | 2 +- .../atmosphere/satwnd.modis_aqua.yaml.j2 | 2 +- .../atmosphere/satwnd.modis_terra.yaml.j2 | 2 +- .../atmosphere/satwnd.seviri_m10.yaml.j2 | 112 +-- .../atmosphere/satwnd.seviri_m11.yaml.j2 | 114 +-- .../atmosphere/satwnd.seviri_m8.yaml.j2 | 109 +-- .../atmosphere/satwnd.seviri_m9.yaml.j2 | 111 +-- .../atmosphere/satwnd.viirs_n20.yaml.j2 | 2 +- .../atmosphere/satwnd.viirs_npp.yaml.j2 | 2 +- ....yaml.j2 => scatwnd.ascat_metop-a.yaml.j2} | 2 + ....yaml.j2 => scatwnd.ascat_metop-b.yaml.j2} | 2 + .../atmosphere/scatwnd.ascat_metop-c.yaml.j2 | 315 ++++++++ .../observations/atmosphere/ssmis_f17.yaml.j2 | 2 + .../observations/atmosphere/ssmis_f18.yaml.j2 | 2 + 38 files changed, 4558 insertions(+), 1178 deletions(-) create mode 100644 parm/jcb-gdas/observation_chronicle/atmosphere/cris-fsr_n21.yaml create mode 100644 parm/jcb-gdas/observation_chronicle/atmosphere/iasi_metop-c.yaml create mode 100644 parm/jcb-gdas/observations/atmosphere/cris-fsr_n21.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/iasi_metop-c.yaml.j2 rename parm/jcb-gdas/observations/atmosphere/{omi_aura.yaml.j2 => ozone.omi_aura.yaml.j2} (71%) create mode 100644 parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_n20.yaml.j2 rename parm/jcb-gdas/observations/atmosphere/{ompsnp_npp.yaml.j2 => ozone.ompsnp_npp.yaml.j2} (95%) create mode 100644 parm/jcb-gdas/observations/atmosphere/ozone.ompstc_n20.yaml.j2 rename parm/jcb-gdas/observations/atmosphere/{ompstc_npp.yaml.j2 => ozone.ompstc_npp.yaml.j2} (96%) create mode 100755 parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-a.yaml.j2 create mode 100755 parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-b.yaml.j2 create mode 100755 parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-c.yaml.j2 rename parm/jcb-gdas/observations/atmosphere/{scatwind_ascat_metop-a.yaml.j2 => scatwnd.ascat_metop-a.yaml.j2} (99%) rename parm/jcb-gdas/observations/atmosphere/{ascatw.ascat_metop-b.yaml.j2 => scatwnd.ascat_metop-b.yaml.j2} (99%) create mode 100644 parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-c.yaml.j2 diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/cris-fsr_n21.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/cris-fsr_n21.yaml new file mode 100644 index 000000000..06b4c357f --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/atmosphere/cris-fsr_n21.yaml @@ -0,0 +1,451 @@ +# Instrument metadata +# ------------------- +commissioned: 2023-08-17T00:00:00 # Source: https://github.com/GEOS-ESM/GEOS_mksi/blob/develop/sidb/active_channels.tbl#L1128 +observer_type: satellite # Type of chronicle to use + +# Instrument initial configuration +# -------------------------------- +channel_variables: + simulated: min + active: min + biascorrtd: min + error: max + error_cld: max + ermax: max + var_b: min + var_pg: min + icld_det: min + icloud: min + iaerosol: min +channel_values: + 19: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 24: [ 1, 1, 1, 0.700, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 26: [ 1, 1, 1, 0.700, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 27: [ 1, -1, 1, 0.700, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 28: [ 1, 1, 1, 0.700, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 31: [ 1, -1, 1, 1.359, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 32: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 33: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 37: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 39: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 42: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 44: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 47: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 49: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 50: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 51: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 52: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 53: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 54: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 55: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 56: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 57: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 58: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 59: [ 1, 1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 60: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 61: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 62: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 63: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 64: [ 1, -1, 1, 0.756, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 65: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 66: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 67: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 68: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 69: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 70: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 71: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 72: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 73: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 74: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 75: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 76: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 77: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 78: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 79: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 80: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 81: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 82: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 83: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 84: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 85: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 86: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 87: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 88: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 89: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 90: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 91: [ 1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 92: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 93: [ 1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 94: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 95: [ 1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 96: [ 1, -1, 1, 0.635, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 97: [ 1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 98: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 99: [ 1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 100: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 101: [ 1, -1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 102: [ 1, -1, 1, 0.735, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 103: [ 1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 104: [ 1, -1, 1, 0.878, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 105: [ 1, 1, 1, 0.450, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 106: [ 1, -1, 1, 0.696, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 107: [ 1, 1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 108: [ 1, -1, 1, 2.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 109: [ 1, 1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 110: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 111: [ 1, 1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 112: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 113: [ 1, 1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 114: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 115: [ 1, 1, 1, 0.400, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 116: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 117: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 118: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 119: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 120: [ 1, -1, 1, 0.701, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 121: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 122: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 123: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 124: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 125: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 126: [ 1, -1, 1, 0.663, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 127: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 128: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 129: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 130: [ 1, -1, 1, 1.083, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 131: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 132: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 133: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 134: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 135: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 136: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 137: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 138: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 139: [ 1, 1, 1, 0.350, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 140: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 141: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 142: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 143: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 144: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 145: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 146: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 147: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 148: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 149: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 150: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 151: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 152: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 153: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 154: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 155: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 156: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 157: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 158: [ 1, -1, 1, 0.773, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 159: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 160: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 161: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 162: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 163: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 164: [ 1, -1, 1, 0.813, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 165: [ 1, -1, 1, 0.907, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 166: [ 1, -1, 1, 0.802, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 167: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 168: [ 1, -1, 1, 1.493, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 169: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 170: [ 1, -1, 1, 0.856, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 171: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 172: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 173: [ 1, -1, 1, 0.600, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 174: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 175: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 176: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 177: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 178: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 179: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 180: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 181: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 182: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 183: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 184: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 185: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 186: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 187: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 188: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 189: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 190: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 191: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 192: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 193: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 194: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 195: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 196: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 197: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 198: [ 1, -1, 1, 2.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 199: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 200: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 208: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 211: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 216: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 224: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 234: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 236: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 238: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 239: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 242: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 246: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 248: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 255: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 264: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 266: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 268: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 275: [ 1, 1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 279: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 283: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 285: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 291: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 295: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 301: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 305: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 311: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 332: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 342: [ 1, 1, 1, 0.300, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 389: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 400: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 402: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 404: [ 1, -1, 1, 2.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 406: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 410: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 427: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 439: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 440: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 441: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 445: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 449: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 455: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 458: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 461: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 464: [ 1, 1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 467: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 470: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 473: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 475: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 482: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 486: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 487: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 490: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 493: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 496: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 499: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 501: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 503: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 505: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 511: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 513: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 514: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 518: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 519: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 520: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 522: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 529: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 534: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 563: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 568: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 575: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 592: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 594: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 596: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 598: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 600: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 602: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 604: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 611: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 614: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 616: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 618: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 620: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 622: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 626: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 631: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 638: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 646: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 648: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 652: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 659: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 673: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 675: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 678: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 684: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 688: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 694: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 700: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 707: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 710: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 713: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, 1, -1, -1 ] + 714: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 718: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 720: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 722: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 725: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 728: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 735: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 742: [ 1, 1, 1, 0.500, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 748: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 753: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 762: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 780: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 784: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 798: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 849: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 860: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 862: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 866: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 874: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 882: [ 1, 1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 890: [ 1, 1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 898: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 906: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 907: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 908: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 914: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 937: [ 1, 1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 972: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 973: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 978: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 980: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 981: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 988: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 995: [ 1, 1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 998: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1000: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1003: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1008: [ 1, 1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 1009: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1010: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1014: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1017: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1018: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1020: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1022: [ 1, 1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 1024: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1026: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1029: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1030: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1032: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1034: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1037: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1038: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1041: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1042: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1044: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1046: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1049: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1050: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1053: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1054: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1058: [ 1, 1, 1, 1.500, 0.000, 0.900, 10.000, 0.000, -1, -1, -1 ] + 1060: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1062: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1064: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1066: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1069: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1076: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1077: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1080: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1086: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1091: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1095: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1101: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1109: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1112: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1121: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1128: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1133: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1163: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1172: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1187: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1189: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1205: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1211: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1219: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1231: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1245: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1271: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1289: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1300: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1313: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1316: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1325: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1329: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1346: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1347: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1473: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1474: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1491: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1499: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1553: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1570: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1596: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1602: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1619: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1624: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1635: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1939: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1940: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1941: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1942: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1943: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1944: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1945: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1946: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1947: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1948: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1949: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1950: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1951: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1952: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1953: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1954: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1955: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1956: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1957: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1958: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1959: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1960: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1961: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1962: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1963: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1964: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1965: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1966: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1967: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1968: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1969: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1970: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1971: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1972: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1973: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1974: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1975: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1976: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1977: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1978: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1979: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1980: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1981: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1982: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1983: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1984: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1985: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1986: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 1987: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2119: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2140: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2143: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2147: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2153: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2158: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2161: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2168: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2171: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2175: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] + 2182: [ 1, -1, 1, 1.000, 0.000, 2.000, 10.000, 0.000, -1, -1, -1 ] diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/iasi_metop-c.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/iasi_metop-c.yaml new file mode 100644 index 000000000..05ccc83a6 --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/atmosphere/iasi_metop-c.yaml @@ -0,0 +1,637 @@ +# Instrument metadata +# ------------------- +commissioned: 2020-05-19T12:00:00 # Source: https://github.com/GEOS-ESM/GEOS_mksi/blob/develop/sidb/active_channels.tbl#L1293 + +observer_type: satellite # Type of chronicle to use + +# Instrument initial configuration +# -------------------------------- +channel_variables: + simulated: min + active: min + biascorrtd: min + error: max + error_cld: max + ermax: max + var_b: min + var_pg: min + icld_det: min + icloud: min + iaerosol: min +channel_values: + 16: [ 1, 1, 1, 1.38, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 29: [ 1, -1, 1, 0.81, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 32: [ 1, -1, 1, 0.75, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 35: [ 1, -1, 1, 0.79, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 38: [ 1, 1, 1, 0.72, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 41: [ 1, -1, 1, 0.74, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 44: [ 1, -1, 1, 0.68, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 47: [ 1, -1, 1, 0.72, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 49: [ 1, 1, 1, 0.65, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 50: [ 1, -1, 1, 0.65, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 51: [ 1, 1, 1, 0.65, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 53: [ 1, -1, 1, 0.69, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 55: [ 1, 1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 56: [ 1, -1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 57: [ 1, 1, 1, 0.65, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 59: [ 1, 1, 1, 0.67, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 61: [ 1, 1, 1, 0.62, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 62: [ 1, -1, 1, 0.61, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 63: [ 1, 1, 1, 0.62, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 66: [ 1, 1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 68: [ 1, -1, 1, 0.59, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 70: [ 1, 1, 1, 0.76, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 72: [ 1, 1, 1, 1.22, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 74: [ 1, 1, 1, 0.78, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 76: [ 1, -1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 78: [ 1, -1, 1, 0.62, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 79: [ 1, 1, 1, 0.61, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 81: [ 1, 1, 1, 0.69, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 82: [ 1, -1, 1, 0.65, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 83: [ 1, 1, 1, 0.59, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 84: [ 1, -1, 1, 0.61, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 85: [ 1, 1, 1, 0.59, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 86: [ 1, -1, 1, 0.68, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 87: [ 1, 1, 1, 0.62, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 89: [ 1, -1, 1, 0.68, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 92: [ 1, -1, 1, 4.38, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 93: [ 1, -1, 1, 3.05, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 95: [ 1, -1, 1, 2.31, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 97: [ 1, -1, 1, 1.56, 0.00, 4.00, 10.00, 0.00, -1, -1, -1 ] + 99: [ 1, -1, 1, 1.33, 0.00, 4.00, 10.00, 0.00, -1, -1, -1 ] + 101: [ 1, -1, 1, 1.58, 0.00, 3.50, 10.00, 0.00, -1, -1, -1 ] + 103: [ 1, -1, 1, 0.93, 0.00, 2.50, 10.00, 0.00, -1, -1, -1 ] + 104: [ 1, 1, 1, 1.67, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 106: [ 1, 1, 1, 0.72, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 109: [ 1, 1, 1, 0.57, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 110: [ 1, -1, 1, 0.58, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 111: [ 1, 1, 1, 0.55, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 113: [ 1, 1, 1, 0.68, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 116: [ 1, 1, 1, 0.59, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 119: [ 1, 1, 1, 0.68, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 122: [ 1, 1, 1, 0.59, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 125: [ 1, 1, 1, 0.65, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 128: [ 1, 1, 1, 0.58, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 131: [ 1, 1, 1, 0.62, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 133: [ 1, 1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 135: [ 1, 1, 1, 0.58, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 138: [ 1, 1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 141: [ 1, 1, 1, 0.55, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 144: [ 1, 1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 146: [ 1, 1, 1, 0.50, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 148: [ 1, 1, 1, 0.82, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 150: [ 1, -1, 1, 0.59, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 151: [ 1, 1, 1, 0.62, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 154: [ 1, 1, 1, 0.51, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 157: [ 1, 1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 159: [ 1, 1, 1, 0.52, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 160: [ 1, -1, 1, 0.51, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 161: [ 1, 1, 1, 0.51, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 163: [ 1, 1, 1, 0.76, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 167: [ 1, 1, 1, 0.52, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 170: [ 1, 1, 1, 0.57, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 173: [ 1, 1, 1, 0.55, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 176: [ 1, 1, 1, 0.69, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 179: [ 1, -1, 1, 0.58, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 180: [ 1, 1, 1, 0.65, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 185: [ 1, 1, 1, 0.61, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 187: [ 1, 1, 1, 0.59, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 191: [ 1, -1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 193: [ 1, 1, 1, 0.76, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 197: [ 1, -1, 1, 0.72, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 199: [ 1, 1, 1, 1.05, 0.00, 3.50, 10.00, 0.00, 1, -1, -1 ] + 200: [ 1, -1, 1, 0.75, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 202: [ 1, -1, 1, 0.51, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 203: [ 1, -1, 1, 0.65, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 205: [ 1, 1, 1, 1.30, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 207: [ 1, 1, 1, 0.69, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 210: [ 1, 1, 1, 0.93, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 212: [ 1, 1, 1, 1.49, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 213: [ 1, -1, 1, 1.12, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 214: [ 1, 1, 1, 0.68, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 217: [ 1, 1, 1, 0.66, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 218: [ 1, -1, 1, 0.67, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 219: [ 1, 1, 1, 0.59, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 222: [ 1, 1, 1, 0.59, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 224: [ 1, 1, 1, 0.69, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 225: [ 1, -1, 1, 0.67, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 226: [ 1, 1, 1, 0.64, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 228: [ 1, -1, 1, 0.62, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 230: [ 1, 1, 1, 0.72, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 231: [ 1, -1, 1, 0.69, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 232: [ 1, 1, 1, 0.66, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 236: [ 1, 1, 1, 0.79, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 237: [ 1, -1, 1, 0.78, 0.00, 2.00, 10.00, 0.00, -1, -1, -1 ] + 239: [ 1, 1, 1, 0.74, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 243: [ 1, 1, 1, 0.88, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 246: [ 1, 1, 1, 0.77, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 249: [ 1, 1, 1, 0.88, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 252: [ 1, 1, 1, 0.86, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 254: [ 1, 1, 1, 1.00, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 259: [ 1, -1, 1, 0.87, 0.00, 2.50, 10.00, 0.00, -1, -1, -1 ] + 260: [ 1, 1, 1, 0.85, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 262: [ 1, 1, 1, 0.88, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 265: [ 1, 1, 1, 0.84, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 267: [ 1, 1, 1, 0.84, 0.00, 3.50, 10.00, 0.00, 1, -1, -1 ] + 269: [ 1, -1, 1, 0.84, 0.00, 2.50, 10.00, 0.00, -1, -1, -1 ] + 275: [ 1, 1, 1, 0.80, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 279: [ 1, -1, 1, 0.80, 0.00, 3.00, 10.00, 0.00, -1, -1, -1 ] + 282: [ 1, 1, 1, 0.87, 0.00, 3.50, 10.00, 0.00, 1, -1, -1 ] + 285: [ 1, -1, 1, 0.98, 0.00, 3.00, 10.00, 0.00, -1, -1, -1 ] + 294: [ 1, 1, 1, 0.52, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 296: [ 1, 1, 1, 0.65, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 299: [ 1, 1, 1, 0.69, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 300: [ 1, -1, 1, 0.61, 0.00, 4.00, 10.00, 0.00, -1, -1, -1 ] + 303: [ 1, 1, 1, 0.60, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 306: [ 1, 1, 1, 0.67, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 309: [ 1, -1, 1, 0.79, 0.00, 4.50, 10.00, 0.00, -1, -1, -1 ] + 313: [ 1, -1, 1, 0.62, 0.00, 4.50, 10.00, 0.00, -1, -1, -1 ] + 320: [ 1, -1, 1, 0.66, 0.00, 4.50, 10.00, 0.00, -1, -1, -1 ] + 323: [ 1, 1, 1, 0.70, 0.00, 4.50, 10.00, 0.00, 1, -1, -1 ] + 326: [ 1, -1, 1, 0.65, 0.00, 4.50, 10.00, 0.00, -1, -1, -1 ] + 327: [ 1, 1, 1, 0.62, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 329: [ 1, 1, 1, 0.61, 0.00, 4.50, 10.00, 0.00, 1, -1, -1 ] + 332: [ 1, -1, 1, 0.62, 0.00, 4.00, 10.00, 0.00, -1, -1, -1 ] + 335: [ 1, 1, 1, 0.53, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 345: [ 1, 1, 1, 0.60, 0.00, 4.50, 10.00, 0.00, 1, -1, -1 ] + 347: [ 1, 1, 1, 0.68, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 350: [ 1, 1, 1, 0.95, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 354: [ 1, 1, 1, 0.63, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 356: [ 1, 1, 1, 0.97, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 360: [ 1, 1, 1, 0.65, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 363: [ 1, -1, 1, 0.98, 0.00, 3.00, 10.00, 0.00, -1, -1, -1 ] + 366: [ 1, 1, 1, 0.58, 0.00, 2.00, 10.00, 0.00, 1, -1, -1 ] + 371: [ 1, 1, 1, 0.73, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 372: [ 1, -1, 1, 0.65, 0.00, 2.50, 10.00, 0.00, -1, -1, -1 ] + 373: [ 1, 1, 1, 0.85, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 375: [ 1, 1, 1, 0.99, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 377: [ 1, 1, 1, 0.76, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 379: [ 1, 1, 1, 0.85, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 381: [ 1, 1, 1, 0.97, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 383: [ 1, 1, 1, 0.77, 0.00, 3.00, 10.00, 0.00, 1, -1, -1 ] + 386: [ 1, 1, 1, 0.62, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 389: [ 1, 1, 1, 0.63, 0.00, 2.50, 10.00, 0.00, 1, -1, -1 ] + 398: [ 1, 1, 1, 1.21, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 401: [ 1, 1, 1, 1.41, 0.00, 4.50, 10.00, 0.00, 1, -1, -1 ] + 404: [ 1, 1, 1, 1.55, 0.00, 4.50, 10.00, 0.00, 1, -1, -1 ] + 405: [ 1, -1, 1, 1.78, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 407: [ 1, 1, 1, 1.35, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 408: [ 1, -1, 1, 1.14, 0.00, 4.00, 10.00, 0.00, -1, -1, -1 ] + 410: [ 1, 1, 1, 1.69, 0.00, 5.00, 10.00, 0.00, 1, -1, -1 ] + 411: [ 1, -1, 1, 1.79, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 414: [ 1, 1, 1, 1.46, 0.00, 5.00, 10.00, 0.00, 1, -1, -1 ] + 416: [ 1, 1, 1, 1.63, 0.00, 5.00, 10.00, 0.00, 1, -1, -1 ] + 418: [ 1, -1, 1, 1.94, 0.00, 5.50, 10.00, 0.00, -1, -1, -1 ] + 423: [ 1, -1, 1, 2.01, 0.00, 5.50, 10.00, 0.00, -1, -1, -1 ] + 426: [ 1, 1, 1, 1.24, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 428: [ 1, 1, 1, 1.76, 0.00, 5.00, 10.00, 0.00, 1, -1, -1 ] + 432: [ 1, 1, 1, 1.26, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 433: [ 1, -1, 1, 1.47, 0.00, 4.50, 10.00, 0.00, -1, -1, -1 ] + 434: [ 1, 1, 1, 1.90, 0.00, 5.50, 10.00, 0.00, 1, -1, -1 ] + 439: [ 1, 1, 1, 1.66, 0.00, 5.50, 10.00, 0.00, 1, -1, -1 ] + 442: [ 1, -1, 1, 2.13, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 445: [ 1, 1, 1, 1.49, 0.00, 4.50, 10.00, 0.00, 1, -1, -1 ] + 450: [ 1, -1, 1, 1.52, 0.00, 4.50, 10.00, 0.00, -1, -1, -1 ] + 457: [ 1, 1, 1, 1.55, 0.00, 4.00, 10.00, 0.00, 1, -1, -1 ] + 459: [ 1, -1, 1, 1.96, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 472: [ 1, -1, 1, 2.31, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 477: [ 1, -1, 1, 2.33, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 483: [ 1, -1, 1, 2.32, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 509: [ 1, -1, 1, 2.31, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 515: [ 1, 1, 1, 2.33, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 546: [ 1, 1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 552: [ 1, 1, 1, 2.33, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 559: [ 1, 1, 1, 1.84, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 566: [ 1, 1, 1, 2.29, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 571: [ 1, 1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 573: [ 1, 1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 578: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 584: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 594: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 625: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 646: [ 1, 1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 662: [ 1, 1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 668: [ 1, 1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 705: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 739: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 756: [ 1, 1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 797: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 867: [ 1, 1, 1, 2.30, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 906: [ 1, 1, 1, 2.15, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 921: [ 1, 1, 1, 2.31, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1027: [ 1, 1, 1, 2.37, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1046: [ 1, 1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1090: [ 1, -1, 1, 2.29, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1098: [ 1, -1, 1, 2.29, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1121: [ 1, 1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1133: [ 1, 1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1173: [ 1, -1, 1, 2.32, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1191: [ 1, 1, 1, 2.32, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1194: [ 1, 1, 1, 2.31, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1222: [ 1, -1, 1, 2.32, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1271: [ 1, 1, 1, 2.32, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1283: [ 1, -1, 1, 2.31, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1338: [ 1, -1, 1, 2.31, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1409: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1414: [ 1, -1, 1, 2.29, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1420: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1424: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1427: [ 1, -1, 1, 2.29, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1430: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1434: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1440: [ 1, -1, 1, 2.25, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1442: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1445: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1450: [ 1, -1, 1, 2.21, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1454: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1460: [ 1, -1, 1, 2.17, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1463: [ 1, -1, 1, 2.18, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1469: [ 1, -1, 1, 2.17, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1474: [ 1, -1, 1, 2.21, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1479: [ 1, 1, 1, 1.99, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1483: [ 1, -1, 1, 2.16, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1487: [ 1, -1, 1, 2.20, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1494: [ 1, -1, 1, 2.13, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1496: [ 1, -1, 1, 2.12, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1502: [ 1, -1, 1, 2.13, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1505: [ 1, -1, 1, 2.10, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1509: [ 1, 1, 1, 2.12, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1510: [ 1, -1, 1, 2.11, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1513: [ 1, 1, 1, 2.09, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1518: [ 1, -1, 1, 2.09, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1521: [ 1, 1, 1, 2.08, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1526: [ 1, -1, 1, 2.09, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1529: [ 1, -1, 1, 2.04, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1532: [ 1, -1, 1, 2.04, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1536: [ 1, 1, 1, 2.10, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1537: [ 1, -1, 1, 2.01, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1541: [ 1, -1, 1, 2.05, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1545: [ 1, -1, 1, 2.03, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1548: [ 1, -1, 1, 2.06, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1553: [ 1, -1, 1, 1.98, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1560: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1568: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1574: [ 1, 1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1579: [ 1, 1, 1, 1.70, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1583: [ 1, -1, 1, 1.76, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1585: [ 1, 1, 1, 1.77, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1587: [ 1, 1, 1, 1.83, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1606: [ 1, -1, 1, 2.04, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1626: [ 1, 1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1639: [ 1, 1, 1, 1.99, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1643: [ 1, 1, 1, 1.99, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1652: [ 1, 1, 1, 2.07, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1658: [ 1, 1, 1, 2.02, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1659: [ 1, -1, 1, 2.04, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1666: [ 1, -1, 1, 2.10, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1671: [ 1, 1, 1, 2.06, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1675: [ 1, -1, 1, 2.18, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1681: [ 1, -1, 1, 2.21, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1694: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1697: [ 1, -1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1710: [ 1, -1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1786: [ 1, 1, 1, 1.98, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1791: [ 1, -1, 1, 2.20, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1805: [ 1, 1, 1, 2.18, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1839: [ 1, -1, 1, 2.18, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1884: [ 1, 1, 1, 2.21, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 1913: [ 1, -1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1946: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1947: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 1991: [ 1, 1, 1, 2.25, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2019: [ 1, 1, 1, 1.80, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2094: [ 1, 1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2119: [ 1, 1, 1, 1.73, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2213: [ 1, 1, 1, 1.73, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2239: [ 1, 1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2271: [ 1, 1, 1, 1.67, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2289: [ 1, -1, 1, 2.21, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2321: [ 1, 1, 1, 1.72, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2333: [ 1, -1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2346: [ 1, -1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2349: [ 1, -1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2352: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2359: [ 1, -1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2367: [ 1, -1, 1, 2.12, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2374: [ 1, -1, 1, 2.17, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2398: [ 1, 1, 1, 1.74, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2426: [ 1, -1, 1, 2.02, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2562: [ 1, -1, 1, 1.88, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2701: [ 1, 1, 1, 1.67, 0.00, 6.00, 10.00, 0.00, 1, -1, -1 ] + 2741: [ 1, -1, 1, 1.73, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2745: [ 1, -1, 1, 1.83, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2760: [ 1, -1, 1, 1.82, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2819: [ 1, -1, 1, 1.73, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2889: [ 1, 1, 1, 1.83, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2907: [ 1, -1, 1, 2.19, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2910: [ 1, -1, 1, 1.84, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2919: [ 1, -1, 1, 1.89, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2921: [ 1, -1, 1, 1.60, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2939: [ 1, -1, 1, 1.71, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2944: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2945: [ 1, -1, 1, 1.85, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2948: [ 1, -1, 1, 1.84, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2951: [ 1, -1, 1, 1.87, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2958: [ 1, 1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2971: [ 1, -1, 1, 1.52, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2977: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2985: [ 1, -1, 1, 1.87, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2988: [ 1, -1, 1, 1.89, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2990: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2991: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 2993: [ 1, 1, 1, 1.93, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3002: [ 1, 1, 1, 1.90, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3008: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3014: [ 1, -1, 1, 1.90, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3027: [ 1, -1, 1, 1.89, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3029: [ 1, -1, 1, 1.89, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3030: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3036: [ 1, -1, 1, 1.90, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3047: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3049: [ 1, 1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3052: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3053: [ 1, -1, 1, 1.93, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3055: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3058: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3064: [ 1, -1, 1, 1.92, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3069: [ 1, -1, 1, 1.77, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3087: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3093: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3098: [ 1, -1, 1, 1.19, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3105: [ 1, 1, 1, 1.96, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3107: [ 1, -1, 1, 1.98, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3110: [ 1, 1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3116: [ 1, -1, 1, 1.55, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3127: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3129: [ 1, -1, 1, 1.92, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3136: [ 1, -1, 1, 1.92, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3146: [ 1, -1, 1, 1.97, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3151: [ 1, -1, 1, 1.93, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3160: [ 1, -1, 1, 1.99, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3165: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3168: [ 1, -1, 1, 1.12, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3175: [ 1, -1, 1, 1.93, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3178: [ 1, -1, 1, 1.92, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3189: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3207: [ 1, -1, 1, 1.85, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3228: [ 1, -1, 1, 1.84, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3244: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3248: [ 1, -1, 1, 1.12, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3252: [ 1, -1, 1, 1.82, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3256: [ 1, -1, 1, 1.82, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3263: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3281: [ 1, -1, 1, 1.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3295: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3303: [ 1, -1, 1, 1.96, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3309: [ 1, -1, 1, 1.21, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3312: [ 1, -1, 1, 1.83, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3322: [ 1, -1, 1, 1.96, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3326: [ 1, -1, 1, 1.36, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3354: [ 1, -1, 1, 1.96, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3366: [ 1, -1, 1, 1.82, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3375: [ 1, -1, 1, 1.92, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3378: [ 1, -1, 1, 1.68, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3411: [ 1, -1, 1, 1.93, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3416: [ 1, -1, 1, 1.23, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3432: [ 1, -1, 1, 1.96, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3438: [ 1, -1, 1, 1.93, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3440: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3442: [ 1, -1, 1, 1.41, 0.00, 5.50, 10.00, 0.00, -1, -1, -1 ] + 3444: [ 1, -1, 1, 1.16, 0.00, 4.50, 10.00, 0.00, -1, -1, -1 ] + 3446: [ 1, -1, 1, 1.60, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3448: [ 1, -1, 1, 1.25, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 3450: [ 1, -1, 1, 1.20, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 3452: [ 1, -1, 1, 1.65, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3454: [ 1, -1, 1, 1.66, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3458: [ 1, -1, 1, 1.87, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3467: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3476: [ 1, -1, 1, 1.96, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3484: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3491: [ 1, -1, 1, 1.25, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 3497: [ 1, -1, 1, 1.93, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3499: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3504: [ 1, -1, 1, 1.70, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3506: [ 1, -1, 1, 0.99, 0.00, 4.00, 10.00, 0.00, -1, -1, -1 ] + 3509: [ 1, -1, 1, 1.81, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3518: [ 1, -1, 1, 1.92, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3527: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3555: [ 1, -1, 1, 1.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3575: [ 1, -1, 1, 1.47, 0.00, 4.50, 10.00, 0.00, -1, -1, -1 ] + 3577: [ 1, -1, 1, 1.15, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3580: [ 1, -1, 1, 1.58, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3582: [ 1, -1, 1, 1.18, 0.00, 4.50, 10.00, 0.00, -1, -1, -1 ] + 3586: [ 1, -1, 1, 1.82, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3589: [ 1, -1, 1, 1.13, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3599: [ 1, -1, 1, 1.83, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3610: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3626: [ 1, -1, 1, 1.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3638: [ 1, -1, 1, 1.27, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 3646: [ 1, -1, 1, 1.91, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3653: [ 1, -1, 1, 1.45, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3658: [ 1, -1, 1, 1.60, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3661: [ 1, -1, 1, 1.29, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 3673: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3689: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3700: [ 1, -1, 1, 1.23, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 3710: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3726: [ 1, -1, 1, 1.21, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 3763: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3814: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3841: [ 1, -1, 1, 1.90, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 3888: [ 1, -1, 1, 1.33, 0.00, 5.00, 10.00, 0.00, -1, -1, -1 ] + 4032: [ 1, -1, 1, 1.75, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4059: [ 1, -1, 1, 2.02, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4068: [ 1, -1, 1, 1.98, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4082: [ 1, -1, 1, 2.03, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4095: [ 1, -1, 1, 1.83, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4160: [ 1, -1, 1, 1.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4234: [ 1, -1, 1, 2.04, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4257: [ 1, -1, 1, 2.02, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4411: [ 1, -1, 1, 1.90, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4498: [ 1, -1, 1, 2.00, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4520: [ 1, -1, 1, 2.02, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4552: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4567: [ 1, -1, 1, 1.93, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4608: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4646: [ 1, -1, 1, 1.95, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4698: [ 1, -1, 1, 1.99, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4808: [ 1, -1, 1, 2.00, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4849: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4920: [ 1, -1, 1, 1.96, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4939: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4947: [ 1, -1, 1, 1.92, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4967: [ 1, -1, 1, 1.88, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4991: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 4996: [ 1, -1, 1, 1.84, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5015: [ 1, -1, 1, 1.87, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5028: [ 1, -1, 1, 1.77, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5056: [ 1, -1, 1, 1.89, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5128: [ 1, -1, 1, 1.89, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5130: [ 1, -1, 1, 1.88, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5144: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5170: [ 1, -1, 1, 1.82, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5178: [ 1, -1, 1, 1.79, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5183: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5188: [ 1, -1, 1, 2.06, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5191: [ 1, -1, 1, 2.33, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5368: [ 1, -1, 1, 1.88, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5371: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5379: [ 1, -1, 1, 1.81, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5381: [ 1, 1, 1, 1.80, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5383: [ 1, -1, 1, 1.80, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5397: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5399: [ 1, 1, 1, 1.90, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5401: [ 1, -1, 1, 2.00, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5403: [ 1, -1, 1, 2.06, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5405: [ 1, -1, 1, 2.10, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5446: [ 1, -1, 1, 2.20, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5455: [ 1, -1, 1, 2.00, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5472: [ 1, -1, 1, 2.16, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5480: [ 1, 1, 1, 1.98, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5483: [ 1, -1, 1, 1.80, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5485: [ 1, -1, 1, 1.80, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5492: [ 1, -1, 1, 1.85, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5497: [ 1, -1, 1, 1.75, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5502: [ 1, -1, 1, 2.04, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5507: [ 1, -1, 1, 2.19, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5509: [ 1, -1, 1, 2.14, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5517: [ 1, -1, 1, 2.19, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5528: [ 1, -1, 1, 1.86, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5558: [ 1, -1, 1, 2.10, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5697: [ 1, -1, 1, 2.11, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5714: [ 1, -1, 1, 2.18, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5749: [ 1, -1, 1, 2.03, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5766: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5785: [ 1, -1, 1, 2.19, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5798: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5799: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5801: [ 1, -1, 1, 2.21, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5817: [ 1, -1, 1, 2.21, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5833: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5834: [ 1, -1, 1, 2.33, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5836: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5849: [ 1, -1, 1, 2.21, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5851: [ 1, -1, 1, 2.12, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5852: [ 1, -1, 1, 2.23, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5865: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5869: [ 1, -1, 1, 2.25, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5881: [ 1, -1, 1, 1.88, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5884: [ 1, -1, 1, 2.26, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5897: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5900: [ 1, -1, 1, 2.36, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5916: [ 1, -1, 1, 2.29, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5932: [ 1, -1, 1, 2.35, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5948: [ 1, -1, 1, 2.30, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5963: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5968: [ 1, -1, 1, 2.08, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5978: [ 1, -1, 1, 2.05, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5988: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5992: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5994: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 5997: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6003: [ 1, -1, 1, 1.97, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6008: [ 1, -1, 1, 2.25, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6023: [ 1, -1, 1, 2.25, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6026: [ 1, -1, 1, 2.25, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6039: [ 1, -1, 1, 2.31, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6053: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6056: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6067: [ 1, -1, 1, 2.13, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6071: [ 1, -1, 1, 2.24, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6082: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6085: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6098: [ 1, -1, 1, 2.41, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6112: [ 1, -1, 1, 2.34, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6126: [ 1, -1, 1, 9.32, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6135: [ 1, -1, 1, 2.28, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6140: [ 1, -1, 1, 2.38, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6149: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6154: [ 1, -1, 1, 2.27, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6158: [ 1, -1, 1, 2.39, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6161: [ 1, -1, 1, 2.11, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6168: [ 1, -1, 1, 2.09, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6174: [ 1, -1, 1, 2.10, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6182: [ 1, -1, 1, 2.06, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6187: [ 1, -1, 1, 2.12, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6205: [ 1, -1, 1, 2.08, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6209: [ 1, -1, 1, 2.00, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6213: [ 1, -1, 1, 1.93, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6317: [ 1, -1, 1, 2.02, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6339: [ 1, -1, 1, 2.55, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6342: [ 1, -1, 1, 1.54, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6366: [ 1, -1, 1, 1.64, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6381: [ 1, -1, 1, 1.51, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6391: [ 1, -1, 1, 1.55, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6489: [ 1, -1, 1, 2.82, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6962: [ 1, -1, 1, 2.92, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6966: [ 1, -1, 1, 2.55, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6970: [ 1, -1, 1, 2.37, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6975: [ 1, -1, 1, 1.85, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6977: [ 1, -1, 1, 1.60, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6982: [ 1, -1, 1, 1.72, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6985: [ 1, -1, 1, 1.74, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6987: [ 1, -1, 1, 1.79, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6989: [ 1, -1, 1, 1.90, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6991: [ 1, -1, 1, 1.94, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6993: [ 1, -1, 1, 2.00, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6995: [ 1, -1, 1, 2.04, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6997: [ 1, -1, 1, 2.08, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 6999: [ 1, -1, 1, 2.12, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7000: [ 1, -1, 1, 2.13, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7004: [ 1, -1, 1, 2.16, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7008: [ 1, -1, 1, 2.18, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7013: [ 1, -1, 1, 2.18, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7016: [ 1, -1, 1, 2.20, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7021: [ 1, -1, 1, 2.20, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7024: [ 1, -1, 1, 2.41, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7027: [ 1, -1, 1, 2.39, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7029: [ 1, -1, 1, 2.38, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7032: [ 1, -1, 1, 2.40, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7038: [ 1, -1, 1, 2.42, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7043: [ 1, -1, 1, 2.41, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7046: [ 1, -1, 1, 2.43, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7049: [ 1, -1, 1, 2.45, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7069: [ 1, -1, 1, 2.43, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7072: [ 1, -1, 1, 2.45, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7076: [ 1, -1, 1, 2.43, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7081: [ 1, -1, 1, 2.40, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7084: [ 1, -1, 1, 2.44, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7089: [ 1, -1, 1, 2.40, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7099: [ 1, -1, 1, 2.42, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7209: [ 1, -1, 1, 2.43, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7222: [ 1, -1, 1, 2.45, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7231: [ 1, -1, 1, 2.45, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7235: [ 1, -1, 1, 2.45, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7247: [ 1, -1, 1, 2.46, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7267: [ 1, -1, 1, 2.45, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7269: [ 1, -1, 1, 2.45, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7284: [ 1, -1, 1, 2.43, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7389: [ 1, -1, 1, 2.51, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7419: [ 1, -1, 1, 2.48, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7423: [ 1, -1, 1, 2.48, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7424: [ 1, -1, 1, 2.53, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7426: [ 1, -1, 1, 2.46, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7428: [ 1, -1, 1, 2.49, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7431: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7436: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7444: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7475: [ 1, -1, 1, 2.52, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7549: [ 1, -1, 1, 2.52, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7584: [ 1, -1, 1, 2.54, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7665: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7666: [ 1, -1, 1, 2.48, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7831: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7836: [ 1, -1, 1, 2.55, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7853: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7865: [ 1, -1, 1, 2.48, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7885: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7888: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7912: [ 1, -1, 1, 2.52, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7950: [ 1, -1, 1, 2.52, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7972: [ 1, -1, 1, 2.48, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7980: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 7995: [ 1, -1, 1, 2.50, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 8007: [ 1, -1, 1, 2.52, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 8015: [ 1, -1, 1, 2.46, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 8055: [ 1, -1, 1, 2.53, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] + 8078: [ 1, -1, 1, 9.00, 0.00, 6.00, 10.00, 0.00, -1, -1, -1 ] diff --git a/parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 index 1ecd386ab..225a485dd 100644 --- a/parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 @@ -54,6 +54,7 @@ name: ObsValue/stationPressure xvals: [80000, 75000, 70000, 65000, 60000, 55000] errors: [110, 120, 120, 120, 120, 1.0e+11] + - filter: Perform Action filter variables: - name: stationPressure @@ -70,6 +71,7 @@ name: ObsValue/stationPressure xvals: [85000, 80000, 75000, 70000, 65000, 60000, 55000] errors: [120, 140, 140, 140, 140, 140, 1.0e+11] + # Initial Error Assignments for SFCSHIP Observations - filter: Perform Action filter variables: @@ -87,6 +89,7 @@ name: ObsValue/stationPressure xvals: [60000, 55000] errors: [130, 1.0e+11] + - filter: Perform Action filter variables: - name: stationPressure @@ -114,6 +117,7 @@ name: ObsValue/stationPressure xvals: [80000, 75000, 70000, 65000, 60000, 55000] errors: [110, 120, 120, 120, 120, 1.0e+11] + # Create PreQC group variable (pqm in GSI read_prepbufr) - filter: Variable Assignment assignments: @@ -286,6 +290,7 @@ - name: ObsValue/stationPressure - name: HofX/stationPressure coefs: [1, -1] + - filter: Variable Assignment assignments: - name: DerivedMetaData/ObsErrorBoundSfcPressure1 diff --git a/parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 index e0737160e..a1640e4ad 100644 --- a/parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: CRIS-FSR N20 + name: CRIS-FSR N20 obsdatain: engine: type: H5File @@ -14,20 +14,22 @@ obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 - simulated variables: [brightness_temperature] + observed variables: [spectralRadiance] + simulated variables: [brightnessTemperature] + derived variables: [brightnessTemperature] channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} # Observation Operator # -------------------- obs operator: name: CRTM - Absorbers: [H2O, O3] - Clouds: [Water, Ice] - Cloud_Fraction: 1.0 + Absorbers: [H2O, O3, CO2] obs options: Sensor_ID: &{{observation_from_jcb}}_sensor_id cris-fsr_n20 EndianType: little_endian CoefficientPath: "{{crtm_coefficient_path}}" + linear obs operator: + Absorbers: [H2O, O3] # Observation Bias Correction (VarBC) # ----------------------------------- @@ -35,7 +37,7 @@ input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" variables without bc: [brightnessTemperature] - channels: &{{observation_from_jcb}}_not_bias_corrected {{ get_satellite_variable(observation_from_jcb, "not_biascorrtd") }} + channels: {{ get_satellite_variable(observation_from_jcb, "not_biascorrtd") }} variational bc: predictors: - name: constant @@ -63,380 +65,306 @@ ratio: 1.1 ratio for small dataset: 2.0 output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" - # - #obs filters: - ## Wavenumber Check - #- filter: BlackList - # filter variables: - # - name: brightness_temperature - # channels: 1972, 1973, 1974, 1975, 1976, 1977, 1978, 1979, 1980, 1981, - # 1982, 1983, 1984, 1985, 1986, 1987, 2119, 2140, 2143, 2147, - # 2153, 2158, 2161, 2168, 2171, 2175, 2182 - # where: - # - variable: - # name: solar_zenith_angle@MetaData - # maxvalue: 88.9999 - # - variable: - # name: water_area_fraction@GeoVaLs - # minvalue: 1.0e-12 - # action: - # name: reject - #- filter: BlackList - # filter variables: - # - name: brightness_temperature - # channels: *{{observation_from_jcb}}_simulated_channels - # action: - # name: inflate error - # inflation variable: - # name: ObsErrorFactorWavenumIR@ObsFunction - # channels: *{{observation_from_jcb}}_simulated_channels - # options: - # channels: *{{observation_from_jcb}}_simulated_channels - ## Observation Range Sanity Check - #- filter: Bounds Check - # filter variables: - # - name: brightness_temperature - # channels: *{{observation_from_jcb}}_simulated_channels - # minvalue: 50.00001 - # maxvalue: 449.99999 - # action: - # name: reject - ## Topography Check - #- filter: BlackList - # filter variables: - # - name: brightness_temperature - # channels: *{{observation_from_jcb}}_simulated_channels - # action: - # name: inflate error - # inflation variable: - # name: ObsErrorFactorTopoRad@ObsFunction - # channels: *{{observation_from_jcb}}_simulated_channels - # options: - # channels: *{{observation_from_jcb}}_simulated_channels - # sensor: *{{observation_from_jcb}}_sensor_id - ## Transmittance Top Check - #- filter: BlackList - # filter variables: - # - name: brightness_temperature - # channels: *{{observation_from_jcb}}_simulated_channels - # action: - # name: inflate error - # inflation variable: - # name: ObsErrorFactorTransmitTopRad@ObsFunction - # channels: *{{observation_from_jcb}}_simulated_channels - # options: - # channels: *{{observation_from_jcb}}_simulated_channels - ## Cloud Detection Check - #- filter: Bounds Check - # filter variables: - # - name: brightness_temperature - # channels: *{{observation_from_jcb}}_simulated_channels - # test variables: - # - name: CloudDetectMinResidualIR@ObsFunction - # channels: *{{observation_from_jcb}}_simulated_channels - # options: - # channels: *{{observation_from_jcb}}_simulated_channels - # use_flag: [ -1, 1, 1, -1, 1, -1, 1, -1, 1, 1, - # 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, -1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, - # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, - # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, - # -1, 1, -1, -1, 1, -1, -1, -1, 1, -1, - # -1, 1, -1, -1, 1, -1, 1, -1, 1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # 1, 1, -1, -1, 1, -1, -1, -1, 1, 1, - # 1, 1, -1, -1, -1, -1, 1, 1, -1, -1, - # -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, - # -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, - # 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, 1, 1, -1, -1, -1, -1, -1, 1, - # -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, - # 1, -1, -1, -1, -1, -1, -1, 1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1] - # use_flag_clddet: [ -1, 1, -1, -1, -1, -1, 1, -1, 1, -1, - # 1, -1, 1, -1, -1, 1, -1, -1, -1, 1, - # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, - # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, - # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, - # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, - # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, - # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, - # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, - # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, - # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, - # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, - # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, - # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, - # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, - # -1, 1, -1, -1, 1, -1, -1, -1, 1, -1, - # -1, 1, -1, -1, 1, -1, 1, -1, 1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, - # 1, 1, -1, -1, -1, -1, 1, 1, -1, -1, - # -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, - # -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1] - # obserr_dtempf: [0.50,2.00,4.00,2.00,4.00] - # maxvalue: 1.0e-12 - # action: - # name: reject - ## Surface Temperature Jacobian Check over Land - #- filter: Bounds Check - # filter variables: - # - name: brightness_temperature - # channels: *{{observation_from_jcb}}_simulated_channels - # where: - # - variable: - # name: water_area_fraction@GeoVaLs - # maxvalue: 0.99 - # test variables: - # - name: brightness_temperature_jacobian_surface_temperature@ObsDiag - # channels: *{{observation_from_jcb}}_simulated_channels - # maxvalue: 0.2 - ## NSST Retrieval Check - #- filter: Bounds Check - # filter variables: - # - name: brightness_temperature - # channels: *{{observation_from_jcb}}_simulated_channels - # test variables: - # - name: NearSSTRetCheckIR@ObsFunction - # channels: *{{observation_from_jcb}}_simulated_channels - # options: - # channels: *{{observation_from_jcb}}_simulated_channels - # use_flag: [ -1, 1, 1, -1, 1, -1, 1, -1, 1, 1, - # 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, -1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, - # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, - # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, - # -1, 1, -1, -1, 1, -1, -1, -1, 1, -1, - # -1, 1, -1, -1, 1, -1, 1, -1, 1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # 1, 1, -1, -1, 1, -1, -1, -1, 1, 1, - # 1, 1, -1, -1, -1, -1, 1, 1, -1, -1, - # -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, - # -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, - # 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, 1, 1, -1, -1, -1, -1, -1, 1, - # -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, - # 1, -1, -1, -1, -1, -1, -1, 1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1 ] - # obserr_demisf: [0.01,0.02,0.03,0.02,0.03] - # obserr_dtempf: [0.50,2.00,4.00,2.00,4.00] - # maxvalue: 1.0e-12 - # action: - # name: reject - ## Surface Jacobians Check - #- filter: BlackList - # filter variables: - # - name: brightness_temperature - # channels: *{{observation_from_jcb}}_simulated_channels - # action: - # name: inflate error - # inflation variable: - # name: ObsErrorFactorSurfJacobianRad@ObsFunction - # channels: *{{observation_from_jcb}}_simulated_channels - # options: - # channels: *{{observation_from_jcb}}_simulated_channels - # obserr_demisf: [0.01, 0.02, 0.03, 0.02, 0.03] - # obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] - ## Gross check - #- filter: Background Check - # filter variables: - # - name: brightness_temperature - # channels: *{{observation_from_jcb}}_simulated_channels - # function absolute threshold: - # - name: ObsErrorBoundIR@ObsFunction - # channels: *{{observation_from_jcb}}_simulated_channels - # options: - # channels: *{{observation_from_jcb}}_simulated_channels - # obserr_bound_latitude: - # name: ObsErrorFactorLatRad@ObsFunction - # options: - # latitude_parameters: [25.0, 0.5, 0.04, 1.0] - # obserr_bound_transmittop: - # name: ObsErrorFactorTransmitTopRad@ObsFunction - # channels: *{{observation_from_jcb}}_simulated_channels - # options: - # channels: *{{observation_from_jcb}}_simulated_channels - # obserr_bound_max: [ 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0 ] - # action: - # name: reject - ## Useflag Check - #- filter: Bounds Check - # filter variables: - # - name: brightness_temperature - # channels: *{{observation_from_jcb}}_simulated_channels - # test variables: - # - name: ChannelUseflagCheckRad@ObsFunction - # channels: *{{observation_from_jcb}}_simulated_channels - # options: - # channels: *{{observation_from_jcb}}_simulated_channels - # use_flag: [ -1, 1, 1, -1, 1, -1, 1, -1, 1, 1, - # 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, -1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, - # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, - # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, - # -1, 1, -1, -1, 1, -1, -1, -1, 1, -1, - # -1, 1, -1, -1, 1, -1, 1, -1, 1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # 1, 1, -1, -1, 1, -1, -1, -1, 1, 1, - # 1, 1, -1, -1, -1, -1, 1, 1, -1, -1, - # -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, - # -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, - # 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, 1, 1, -1, -1, -1, -1, -1, 1, - # -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, - # 1, -1, -1, -1, -1, -1, -1, 1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1 ] - # minvalue: 1.0e-12 - # action: - # name: reject + # Observation Pre Filters (QC) + # ---------------------------- + obs pre filters: + # Step 1: Create Derived Variables + # Assign channel wavenumbers in m-1 + - filter: Variable Assignment + assignments: + - name: MetaData/sensorCentralWavenumber + type: float + channels: &{{observation_from_jcb}}_band1 19, 24, 26, 27, 28, 31, 32, 33, 37, 39, 42, 44, 47, 49, 50, + 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, + 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, + 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, + 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, + 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, + 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, + 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, + 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, + 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, + 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 208, + 211, 216, 224, 234, 236, 238, 239, 242, 246, 248, 255, 264, 266, 268, + 275, 279, 283, 285, 291, 295, 301, 305, 311, 332, 342, 389, 400, 402, + 404, 406, 410, 427, 439, 440, 441, 445, 449, 455, 458, 461, 464, 467, + 470, 473, 475, 482, 486, 487, 490, 493, 496, 499, 501, 503, 505, 511, + 513, 514, 518, 519, 520, 522, 529, 534, 563, 568, 575, 592, 594, 596, + 598, 600, 602, 604, 611, 614, 616, 618, 620, 622, 626, 631, 638, 646, + 648, 652, 659, 673, 675, 678, 684, 688, 694, 700, 707, 710, 713 + function: + name: ObsFunction/LinearCombination + options: + variables: + - name: ObsValue/spectralRadiance + channels: *{{observation_from_jcb}}_band1 + coefs: [62.5] + intercept: 64937.5 + use channel numbers: true + + - filter: Variable Assignment + assignments: + - name: MetaData/sensorCentralWavenumber + type: float + channels: &{{observation_from_jcb}}_band2 714, 718, 720, 722, 725, 728, 735, 742, 748, 753, 762, 780, 784, 798, 849, + 860, 862, 866, 874, 882, 890, 898, 906, 907, 908, 914, 937, 972, 973, + 978, 980, 981, 988, 995, 998, 1000, 1003, 1008, 1009, 1010, 1014, 1017, + 1018, 1020, 1022, 1024, 1026, 1029, 1030, 1032, 1034, 1037, 1038, 1041, + 1042, 1044, 1046, 1049, 1050, 1053, 1054, 1058, 1060, 1062, 1064, 1066, + 1069, 1076, 1077, 1080, 1086, 1091, 1095, 1101, 1109, 1112, 1121, 1128, + 1133, 1163, 1172, 1187, 1189, 1205, 1211, 1219, 1231, 1245, 1271, 1289, + 1300, 1313, 1316, 1325, 1329, 1346, 1347, 1473, 1474, 1491, 1499, 1553, + 1570 + function: + name: ObsFunction/LinearCombination + options: + variables: + - name: ObsValue/spectralRadiance + channels: *{{observation_from_jcb}}_band2 + coefs: [62.5] + intercept: 76375 + use channel numbers: true + + - filter: Variable Assignment + assignments: + - name: MetaData/sensorCentralWavenumber + type: float + channels: &{{observation_from_jcb}}_band3 1596, 1602, 1619, 1624, 1635, 1939, 1940, 1941, 1942, 1943, 1944, + 1945, 1946, 1947, 1948, 1949, 1950, 1951, 1952, 1953, 1954, 1955, 1956, + 1957, 1958, 1959, 1960, 1961, 1962, 1963, 1964, 1965, 1966, 1967, 1968, + 1969, 1970, 1971, 1972, 1973, 1974, 1975, 1976, 1977, 1978, 1979, 1980, + 1981, 1982, 1983, 1984, 1985, 1986, 1987, 2119, 2140, 2143, 2147, 2153, + 2158, 2161, 2168, 2171, 2175, 2182 + function: + name: ObsFunction/LinearCombination + options: + variables: + - name: ObsValue/spectralRadiance + channels: *{{observation_from_jcb}}_band3 + coefs: [62.5] + intercept: 116812.5 + use channel numbers: true + + # Step 2: Transform radiance to brightness temperature + - filter: Variable Transforms + Transform: SatBrightnessTempFromRad + transform from: + name: ObsValue/spectralRadiance + channels: *{{observation_from_jcb}}_simulated_channels + spectral variable: + name: MetaData/sensorCentralWavenumber + channels: *{{observation_from_jcb}}_simulated_channels + radiance units: wavenumber + planck1: 1.191042953e-16 + planck2: 1.4387774e-2 + + # Step 3: Assign observation error + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: assign error + error parameter vector: &{{observation_from_jcb}}_oberr [{{ get_satellite_variable(observation_from_jcb, "error") }}] + + # Step 4: Create cloudFree variable + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/cloudFree + type: int + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsValue/cloudCoverTotal + coefs: [-100] + intercept: 100 + + # Step 5: Data Thinning + - filter: Gaussian Thinning + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + horizontal_mesh: 145 + use_reduced_horizontal_grid: true + distance_norm: geodesic + priority_variable: DerivedMetaData/cloudFree + action: + name: reduce obs space + + obs post filters: + # Step 6: Wavenumber Check + - filter: RejectList + filter variables: + - name: brightnessTemperature + channels: 1972, 1973, 1974, 1975, 1976, 1977, 1978, 1979, 1980, 1981, + 1982, 1983, 1984, 1985, 1986, 1987, 2119, 2140, 2143, 2147, + 2153, 2158, 2161, 2168, 2171, 2175, 2182 + where: + - variable: + name: MetaData/solarZenithAngle + maxvalue: 88.9999 + - variable: + name: GeoVaLs/water_area_fraction + minvalue: 1.0e-12 + action: + name: reject + + # Step 7: Observation Error Inflation based on Wavenumber + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorWavenumIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 8: Observation Range Sanity Check + # Valid range: (50, 550) + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + minvalue: 50.00001 + maxvalue: 549.99999 + action: + name: reject + + # Step 9: Topography Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorTopoRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + sensor: *{{observation_from_jcb}}_sensor_id + + # Step 10: Transmittance Top Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorTransmitTopRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 11: Cloud Detection Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/CloudDetectMinResidualIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: &{{observation_from_jcb}}_active_channels [{{ get_satellite_variable(observation_from_jcb, "active") }}] + use_flag_clddet: [{{ get_satellite_variable(observation_from_jcb, "icld_det") }}] + obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] + error parameter vector: *{{observation_from_jcb}}_oberr + maxvalue: 1.0e-12 + action: + name: reject + + # Step 12: Surface Temperature Jacobian Check over Land + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + test variables: + - name: ObsDiag/brightness_temperature_jacobian_skin_temperature_at_surface + channels: *{{observation_from_jcb}}_simulated_channels + maxvalue: 0.2 + + # Step 13: NSST Retrieval Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/NearSSTRetCheckIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: *{{observation_from_jcb}}_active_channels + obserr_demisf: [0.01, 0.02, 0.03, 0.02, 0.03] + obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] + maxvalue: 1.0e-12 + action: + name: reject + + # Step 14: Surface Jacobians Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorSurfJacobianRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_demisf: [0.01, 0.02, 0.03, 0.02, 0.03] + obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] + sensor: *{{observation_from_jcb}}_sensor_id + + # Step 15: Gross check + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + function absolute threshold: + - name: ObsFunction/ObsErrorBoundIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_latitude: + name: ObsFunction/ObsErrorFactorLatRad + options: + latitude_parameters: [25.0, 0.5, 0.04, 1.0] + obserr_bound_transmittop: + name: ObsFunction/ObsErrorFactorTransmitTopRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_max: [{{ get_satellite_variable(observation_from_jcb, "ermax") }}] + error parameter vector: *{{observation_from_jcb}}_oberr + action: + name: reject + + # Step 16: Useflag Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/ChannelUseflagCheckRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: *{{observation_from_jcb}}_active_channels + minvalue: 1.0e-12 + action: + name: reject # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/cris-fsr_n21.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/cris-fsr_n21.yaml.j2 new file mode 100644 index 000000000..d9737d191 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/cris-fsr_n21.yaml.j2 @@ -0,0 +1,376 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: CRIS-FSR N21 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + observed variables: [spectralRadiance] + simulated variables: [brightnessTemperature] + derived variables: [brightnessTemperature] + channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} + + # Observation Operator + # -------------------- + obs operator: + name: CRTM + Absorbers: [H2O, O3, CO2] + obs options: + Sensor_ID: &{{observation_from_jcb}}_sensor_id cris-fsr_n21 + EndianType: little_endian + CoefficientPath: "{{crtm_coefficient_path}}" + linear obs operator: + Absorbers: [H2O, O3] + + # Observation Bias Correction (VarBC) + # ----------------------------------- + obs bias: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" + variables without bc: [brightnessTemperature] + channels: {{ get_satellite_variable(observation_from_jcb, "not_biascorrtd") }} + variational bc: + predictors: + - name: constant + - name: lapseRate + order: 2 + tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" + - name: lapseRate + tlapse: *{{observation_from_jcb}}_tlapse + - name: emissivityJacobian + - name: sensorScanAngle + order: 4 + - name: sensorScanAngle + order: 3 + - name: sensorScanAngle + order: 2 + - name: sensorScanAngle + covariance: + minimal required obs number: 20 + variance range: [1.0e-6, 10.0] + step size: 1.0e-4 + largest analysis variance: 10000.0 + prior: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiascovin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovin_suffix}}" + inflation: + ratio: 1.1 + ratio for small dataset: 2.0 + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" + + # Observation Pre Filters (QC) + # ---------------------------- + obs pre filters: + # Step 1: Create Derived Variables + # Assign channel wavenumbers in m-1 + - filter: Variable Assignment + assignments: + - name: MetaData/sensorCentralWavenumber + type: float + channels: &{{observation_from_jcb}}_band1 19, 24, 26, 27, 28, 31, 32, 33, 37, 39, 42, 44, 47, 49, 50, + 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, + 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, + 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, + 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, + 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, + 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, + 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, + 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, + 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, + 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 208, + 211, 216, 224, 234, 236, 238, 239, 242, 246, 248, 255, 264, 266, 268, + 275, 279, 283, 285, 291, 295, 301, 305, 311, 332, 342, 389, 400, 402, + 404, 406, 410, 427, 439, 440, 441, 445, 449, 455, 458, 461, 464, 467, + 470, 473, 475, 482, 486, 487, 490, 493, 496, 499, 501, 503, 505, 511, + 513, 514, 518, 519, 520, 522, 529, 534, 563, 568, 575, 592, 594, 596, + 598, 600, 602, 604, 611, 614, 616, 618, 620, 622, 626, 631, 638, 646, + 648, 652, 659, 673, 675, 678, 684, 688, 694, 700, 707, 710, 713 + function: + name: ObsFunction/LinearCombination + options: + variables: + - name: ObsValue/spectralRadiance + channels: *{{observation_from_jcb}}_band1 + coefs: [62.5] + intercept: 64937.5 + use channel numbers: true + + - filter: Variable Assignment + assignments: + - name: MetaData/sensorCentralWavenumber + type: float + channels: &{{observation_from_jcb}}_band2 714, 718, 720, 722, 725, 728, 735, 742, 748, 753, 762, 780, 784, 798, 849, + 860, 862, 866, 874, 882, 890, 898, 906, 907, 908, 914, 937, 972, 973, + 978, 980, 981, 988, 995, 998, 1000, 1003, 1008, 1009, 1010, 1014, 1017, + 1018, 1020, 1022, 1024, 1026, 1029, 1030, 1032, 1034, 1037, 1038, 1041, + 1042, 1044, 1046, 1049, 1050, 1053, 1054, 1058, 1060, 1062, 1064, 1066, + 1069, 1076, 1077, 1080, 1086, 1091, 1095, 1101, 1109, 1112, 1121, 1128, + 1133, 1163, 1172, 1187, 1189, 1205, 1211, 1219, 1231, 1245, 1271, 1289, + 1300, 1313, 1316, 1325, 1329, 1346, 1347, 1473, 1474, 1491, 1499, 1553, + 1570 + function: + name: ObsFunction/LinearCombination + options: + variables: + - name: ObsValue/spectralRadiance + channels: *{{observation_from_jcb}}_band2 + coefs: [62.5] + intercept: 76375 + use channel numbers: true + + - filter: Variable Assignment + assignments: + - name: MetaData/sensorCentralWavenumber + type: float + channels: &{{observation_from_jcb}}_band3 1596, 1602, 1619, 1624, 1635, 1939, 1940, 1941, 1942, 1943, 1944, + 1945, 1946, 1947, 1948, 1949, 1950, 1951, 1952, 1953, 1954, 1955, 1956, + 1957, 1958, 1959, 1960, 1961, 1962, 1963, 1964, 1965, 1966, 1967, 1968, + 1969, 1970, 1971, 1972, 1973, 1974, 1975, 1976, 1977, 1978, 1979, 1980, + 1981, 1982, 1983, 1984, 1985, 1986, 1987, 2119, 2140, 2143, 2147, 2153, + 2158, 2161, 2168, 2171, 2175, 2182 + function: + name: ObsFunction/LinearCombination + options: + variables: + - name: ObsValue/spectralRadiance + channels: *{{observation_from_jcb}}_band3 + coefs: [62.5] + intercept: 116812.5 + use channel numbers: true + + # Step 2: Transform radiance to brightness temperature + - filter: Variable Transforms + Transform: SatBrightnessTempFromRad + transform from: + name: ObsValue/spectralRadiance + channels: *{{observation_from_jcb}}_simulated_channels + spectral variable: + name: MetaData/sensorCentralWavenumber + channels: *{{observation_from_jcb}}_simulated_channels + radiance units: wavenumber + planck1: 1.191042953e-16 + planck2: 1.4387774e-2 + + # Step 3: Assign observation error + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: assign error + error parameter vector: &{{observation_from_jcb}}_oberr [{{ get_satellite_variable(observation_from_jcb, "error") }}] + + # Step 4: Create cloudFree variable + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/cloudFree + type: int + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsValue/cloudCoverTotal + coefs: [-100] + intercept: 100 + + # Step 5: Data Thinning + - filter: Gaussian Thinning + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + horizontal_mesh: 145 + use_reduced_horizontal_grid: true + distance_norm: geodesic + priority_variable: DerivedMetaData/cloudFree + action: + name: reduce obs space + + obs post filters: + # Step 6: Wavenumber Check + - filter: RejectList + filter variables: + - name: brightnessTemperature + channels: 1972, 1973, 1974, 1975, 1976, 1977, 1978, 1979, 1980, 1981, + 1982, 1983, 1984, 1985, 1986, 1987, 2119, 2140, 2143, 2147, + 2153, 2158, 2161, 2168, 2171, 2175, 2182 + where: + - variable: + name: MetaData/solarZenithAngle + maxvalue: 88.9999 + - variable: + name: GeoVaLs/water_area_fraction + minvalue: 1.0e-12 + action: + name: reject + + # Step 7: Observation Error Inflation based on Wavenumber + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorWavenumIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 8: Observation Range Sanity Check + # Valid range: (50, 550) + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + minvalue: 50.00001 + maxvalue: 549.99999 + action: + name: reject + + # Step 9: Topography Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorTopoRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + sensor: *{{observation_from_jcb}}_sensor_id + + # Step 10: Transmittance Top Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorTransmitTopRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 11: Cloud Detection Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/CloudDetectMinResidualIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: &{{observation_from_jcb}}_active_channels [{{ get_satellite_variable(observation_from_jcb, "active") }}] + use_flag_clddet: [{{ get_satellite_variable(observation_from_jcb, "icld_det") }}] + obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] + error parameter vector: *{{observation_from_jcb}}_oberr + maxvalue: 1.0e-12 + action: + name: reject + + # Step 12: Surface Temperature Jacobian Check over Land + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + test variables: + - name: ObsDiag/brightness_temperature_jacobian_skin_temperature_at_surface + channels: *{{observation_from_jcb}}_simulated_channels + maxvalue: 0.2 + + # Step 13: NSST Retrieval Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/NearSSTRetCheckIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: *{{observation_from_jcb}}_active_channels + obserr_demisf: [0.01, 0.02, 0.03, 0.02, 0.03] + obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] + maxvalue: 1.0e-12 + action: + name: reject + + # Step 14: Surface Jacobians Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorSurfJacobianRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_demisf: [0.01, 0.02, 0.03, 0.02, 0.03] + obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] + sensor: *{{observation_from_jcb}}_sensor_id + + # Step 15: Gross check + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + function absolute threshold: + - name: ObsFunction/ObsErrorBoundIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_latitude: + name: ObsFunction/ObsErrorFactorLatRad + options: + latitude_parameters: [25.0, 0.5, 0.04, 1.0] + obserr_bound_transmittop: + name: ObsFunction/ObsErrorFactorTransmitTopRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_max: [{{ get_satellite_variable(observation_from_jcb, "ermax") }}] + error parameter vector: *{{observation_from_jcb}}_oberr + action: + name: reject + + # Step 16: Useflag Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/ChannelUseflagCheckRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: *{{observation_from_jcb}}_active_channels + minvalue: 1.0e-12 + action: + name: reject + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/cris-fsr_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/cris-fsr_npp.yaml.j2 index 5e3cae3b7..47603300c 100644 --- a/parm/jcb-gdas/observations/atmosphere/cris-fsr_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/cris-fsr_npp.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: CRIS-FSR NPP + name: CRIS-FSR NPP obsdatain: engine: type: H5File @@ -14,26 +14,30 @@ obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 - simulated variables: [brightness_temperature] + observed variables: [spectralRadiance] + simulated variables: [brightnessTemperature] + derived variables: [brightnessTemperature] channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} # Observation Operator # -------------------- obs operator: name: CRTM - Absorbers: [H2O, O3] - Clouds: [Water, Ice] - Cloud_Fraction: 1.0 + Absorbers: [H2O, O3, CO2] obs options: Sensor_ID: &{{observation_from_jcb}}_sensor_id cris-fsr_npp EndianType: little_endian CoefficientPath: "{{crtm_coefficient_path}}" + linear obs operator: + Absorbers: [H2O, O3] # Observation Bias Correction (VarBC) # ----------------------------------- obs bias: input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" + variables without bc: [brightnessTemperature] + channels: {{ get_satellite_variable(observation_from_jcb, "not_biascorrtd") }} variational bc: predictors: - name: constant @@ -61,379 +65,306 @@ ratio: 1.1 ratio for small dataset: 2.0 output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" - #obs filters: - ## Wavenumber Check - #- filter: BlackList - # filter variables: - # - name: brightness_temperature - # channels: 1972, 1973, 1974, 1975, 1976, 1977, 1978, 1979, 1980, 1981, - # 1982, 1983, 1984, 1985, 1986, 1987, 2119, 2140, 2143, 2147, - # 2153, 2158, 2161, 2168, 2171, 2175, 2182 - # where: - # - variable: - # name: solar_zenith_angle@MetaData - # maxvalue: 88.9999 - # - variable: - # name: water_area_fraction@GeoVaLs - # minvalue: 1.0e-12 - # action: - # name: reject - #- filter: BlackList - # filter variables: - # - name: brightness_temperature - # channels: *{{observation_from_jcb}}_simulated_channels - # action: - # name: inflate error - # inflation variable: - # name: ObsErrorFactorWavenumIR@ObsFunction - # channels: *{{observation_from_jcb}}_simulated_channels - # options: - # channels: *{{observation_from_jcb}}_simulated_channels - ## Observation Range Sanity Check - #- filter: Bounds Check - # filter variables: - # - name: brightness_temperature - # channels: *{{observation_from_jcb}}_simulated_channels - # minvalue: 50.00001 - # maxvalue: 449.99999 - # action: - # name: reject - ## Topography Check - #- filter: BlackList - # filter variables: - # - name: brightness_temperature - # channels: *{{observation_from_jcb}}_simulated_channels - # action: - # name: inflate error - # inflation variable: - # name: ObsErrorFactorTopoRad@ObsFunction - # channels: *{{observation_from_jcb}}_simulated_channels - # options: - # channels: *{{observation_from_jcb}}_simulated_channels - # sensor: *{{observation_from_jcb}}_sensor_id - ## Transmittance Top Check - #- filter: BlackList - # filter variables: - # - name: brightness_temperature - # channels: *{{observation_from_jcb}}_simulated_channels - # action: - # name: inflate error - # inflation variable: - # name: ObsErrorFactorTransmitTopRad@ObsFunction - # channels: *{{observation_from_jcb}}_simulated_channels - # options: - # channels: *{{observation_from_jcb}}_simulated_channels - ## Cloud Detection Check - #- filter: Bounds Check - # filter variables: - # - name: brightness_temperature - # channels: *{{observation_from_jcb}}_simulated_channels - # test variables: - # - name: CloudDetectMinResidualIR@ObsFunction - # channels: *{{observation_from_jcb}}_simulated_channels - # options: - # channels: *{{observation_from_jcb}}_simulated_channels - # use_flag: [ -1, 1, 1, -1, 1, -1, 1, -1, 1, 1, - # 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, -1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, - # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, - # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, - # -1, 1, -1, -1, 1, -1, -1, -1, 1, -1, - # -1, 1, -1, -1, 1, -1, 1, -1, 1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # 1, 1, -1, -1, 1, -1, -1, -1, 1, 1, - # 1, 1, -1, -1, -1, -1, 1, 1, -1, -1, - # -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, - # -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, - # 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, 1, 1, -1, -1, -1, -1, -1, 1, - # -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, - # 1, -1, -1, -1, -1, -1, -1, 1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1] - # use_flag_clddet: [ -1, 1, -1, -1, -1, -1, 1, -1, 1, -1, - # 1, -1, 1, -1, -1, 1, -1, -1, -1, 1, - # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, - # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, - # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, - # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, - # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, - # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, - # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, - # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, - # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, - # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, - # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, - # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, - # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, - # -1, 1, -1, -1, 1, -1, -1, -1, 1, -1, - # -1, 1, -1, -1, 1, -1, 1, -1, 1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, - # 1, 1, -1, -1, -1, -1, 1, 1, -1, -1, - # -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, - # -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1] - # obserr_dtempf: [0.50,2.00,4.00,2.00,4.00] - # maxvalue: 1.0e-12 - # action: - # name: reject - ## Surface Temperature Jacobian Check over Land - #- filter: Bounds Check - # filter variables: - # - name: brightness_temperature - # channels: *{{observation_from_jcb}}_simulated_channels - # where: - # - variable: - # name: water_area_fraction@GeoVaLs - # maxvalue: 0.99 - # test variables: - # - name: brightness_temperature_jacobian_surface_temperature@ObsDiag - # channels: *{{observation_from_jcb}}_simulated_channels - # maxvalue: 0.2 - ## NSST Retrieval Check - #- filter: Bounds Check - # filter variables: - # - name: brightness_temperature - # channels: *{{observation_from_jcb}}_simulated_channels - # test variables: - # - name: NearSSTRetCheckIR@ObsFunction - # channels: *{{observation_from_jcb}}_simulated_channels - # options: - # channels: *{{observation_from_jcb}}_simulated_channels - # use_flag: [ -1, 1, 1, -1, 1, -1, 1, -1, 1, 1, - # 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, -1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, - # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, - # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, - # -1, 1, -1, -1, 1, -1, -1, -1, 1, -1, - # -1, 1, -1, -1, 1, -1, 1, -1, 1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # 1, 1, -1, -1, 1, -1, -1, -1, 1, 1, - # 1, 1, -1, -1, -1, -1, 1, 1, -1, -1, - # -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, - # -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, - # 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, 1, 1, -1, -1, -1, -1, -1, 1, - # -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, - # 1, -1, -1, -1, -1, -1, -1, 1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1 ] - # obserr_demisf: [0.01,0.02,0.03,0.02,0.03] - # obserr_dtempf: [0.50,2.00,4.00,2.00,4.00] - # maxvalue: 1.0e-12 - # action: - # name: reject - ## Surface Jacobians Check - #- filter: BlackList - # filter variables: - # - name: brightness_temperature - # channels: *{{observation_from_jcb}}_simulated_channels - # action: - # name: inflate error - # inflation variable: - # name: ObsErrorFactorSurfJacobianRad@ObsFunction - # channels: *{{observation_from_jcb}}_simulated_channels - # options: - # channels: *{{observation_from_jcb}}_simulated_channels - # obserr_demisf: [0.01, 0.02, 0.03, 0.02, 0.03] - # obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] - ## Gross check - #- filter: Background Check - # filter variables: - # - name: brightness_temperature - # channels: *{{observation_from_jcb}}_simulated_channels - # function absolute threshold: - # - name: ObsErrorBoundIR@ObsFunction - # channels: *{{observation_from_jcb}}_simulated_channels - # options: - # channels: *{{observation_from_jcb}}_simulated_channels - # obserr_bound_latitude: - # name: ObsErrorFactorLatRad@ObsFunction - # options: - # latitude_parameters: [25.0, 0.5, 0.04, 1.0] - # obserr_bound_transmittop: - # name: ObsErrorFactorTransmitTopRad@ObsFunction - # channels: *{{observation_from_jcb}}_simulated_channels - # options: - # channels: *{{observation_from_jcb}}_simulated_channels - # obserr_bound_max: [ 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, - # 2.0 ] - # action: - # name: reject - ## Useflag Check - #- filter: Bounds Check - # filter variables: - # - name: brightness_temperature - # channels: *{{observation_from_jcb}}_simulated_channels - # test variables: - # - name: ChannelUseflagCheckRad@ObsFunction - # channels: *{{observation_from_jcb}}_simulated_channels - # options: - # channels: *{{observation_from_jcb}}_simulated_channels - # use_flag: [ -1, 1, 1, -1, 1, -1, 1, -1, 1, 1, - # 1, 1, 1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, -1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, 1, -1, 1, -1, 1, - # -1, 1, -1, 1, -1, -1, -1, 1, -1, -1, - # -1, 1, -1, -1, -1, 1, -1, -1, -1, 1, - # -1, -1, -1, 1, -1, -1, -1, 1, -1, -1, - # -1, 1, -1, -1, 1, -1, -1, -1, 1, -1, - # -1, 1, -1, -1, 1, -1, 1, -1, 1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # 1, 1, -1, -1, 1, -1, -1, -1, 1, 1, - # 1, 1, -1, -1, -1, -1, 1, 1, -1, -1, - # -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, - # -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, 1, 1, -1, -1, -1, -1, -1, -1, -1, - # 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, 1, 1, -1, -1, -1, -1, -1, 1, - # -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, - # 1, -1, -1, -1, -1, -1, -1, 1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, - # -1 ] - # minvalue: 1.0e-12 - # action: - # name: reject + # Observation Pre Filters (QC) + # ---------------------------- + obs pre filters: + # Step 1: Create Derived Variables + # Assign channel wavenumbers in m-1 + - filter: Variable Assignment + assignments: + - name: MetaData/sensorCentralWavenumber + type: float + channels: &{{observation_from_jcb}}_band1 19, 24, 26, 27, 28, 31, 32, 33, 37, 39, 42, 44, 47, 49, 50, + 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, + 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, + 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, + 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, + 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, + 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, + 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, + 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, + 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, + 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 208, + 211, 216, 224, 234, 236, 238, 239, 242, 246, 248, 255, 264, 266, 268, + 275, 279, 283, 285, 291, 295, 301, 305, 311, 332, 342, 389, 400, 402, + 404, 406, 410, 427, 439, 440, 441, 445, 449, 455, 458, 461, 464, 467, + 470, 473, 475, 482, 486, 487, 490, 493, 496, 499, 501, 503, 505, 511, + 513, 514, 518, 519, 520, 522, 529, 534, 563, 568, 575, 592, 594, 596, + 598, 600, 602, 604, 611, 614, 616, 618, 620, 622, 626, 631, 638, 646, + 648, 652, 659, 673, 675, 678, 684, 688, 694, 700, 707, 710, 713 + function: + name: ObsFunction/LinearCombination + options: + variables: + - name: ObsValue/spectralRadiance + channels: *{{observation_from_jcb}}_band1 + coefs: [62.5] + intercept: 64937.5 + use channel numbers: true + + - filter: Variable Assignment + assignments: + - name: MetaData/sensorCentralWavenumber + type: float + channels: &{{observation_from_jcb}}_band2 714, 718, 720, 722, 725, 728, 735, 742, 748, 753, 762, 780, 784, 798, 849, + 860, 862, 866, 874, 882, 890, 898, 906, 907, 908, 914, 937, 972, 973, + 978, 980, 981, 988, 995, 998, 1000, 1003, 1008, 1009, 1010, 1014, 1017, + 1018, 1020, 1022, 1024, 1026, 1029, 1030, 1032, 1034, 1037, 1038, 1041, + 1042, 1044, 1046, 1049, 1050, 1053, 1054, 1058, 1060, 1062, 1064, 1066, + 1069, 1076, 1077, 1080, 1086, 1091, 1095, 1101, 1109, 1112, 1121, 1128, + 1133, 1163, 1172, 1187, 1189, 1205, 1211, 1219, 1231, 1245, 1271, 1289, + 1300, 1313, 1316, 1325, 1329, 1346, 1347, 1473, 1474, 1491, 1499, 1553, + 1570 + function: + name: ObsFunction/LinearCombination + options: + variables: + - name: ObsValue/spectralRadiance + channels: *{{observation_from_jcb}}_band2 + coefs: [62.5] + intercept: 76375 + use channel numbers: true + + - filter: Variable Assignment + assignments: + - name: MetaData/sensorCentralWavenumber + type: float + channels: &{{observation_from_jcb}}_band3 1596, 1602, 1619, 1624, 1635, 1939, 1940, 1941, 1942, 1943, 1944, + 1945, 1946, 1947, 1948, 1949, 1950, 1951, 1952, 1953, 1954, 1955, 1956, + 1957, 1958, 1959, 1960, 1961, 1962, 1963, 1964, 1965, 1966, 1967, 1968, + 1969, 1970, 1971, 1972, 1973, 1974, 1975, 1976, 1977, 1978, 1979, 1980, + 1981, 1982, 1983, 1984, 1985, 1986, 1987, 2119, 2140, 2143, 2147, 2153, + 2158, 2161, 2168, 2171, 2175, 2182 + function: + name: ObsFunction/LinearCombination + options: + variables: + - name: ObsValue/spectralRadiance + channels: *{{observation_from_jcb}}_band3 + coefs: [62.5] + intercept: 116812.5 + use channel numbers: true + + # Step 2: Transform radiance to brightness temperature + - filter: Variable Transforms + Transform: SatBrightnessTempFromRad + transform from: + name: ObsValue/spectralRadiance + channels: *{{observation_from_jcb}}_simulated_channels + spectral variable: + name: MetaData/sensorCentralWavenumber + channels: *{{observation_from_jcb}}_simulated_channels + radiance units: wavenumber + planck1: 1.191042953e-16 + planck2: 1.4387774e-2 + + # Step 3: Assign observation error + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: assign error + error parameter vector: &{{observation_from_jcb}}_oberr [{{ get_satellite_variable(observation_from_jcb, "error") }}] + + # Step 4: Create cloudFree variable + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/cloudFree + type: int + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsValue/cloudCoverTotal + coefs: [-100] + intercept: 100 + + # Step 5: Data Thinning + - filter: Gaussian Thinning + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + horizontal_mesh: 145 + use_reduced_horizontal_grid: true + distance_norm: geodesic + priority_variable: DerivedMetaData/cloudFree + action: + name: reduce obs space + + obs post filters: + # Step 6: Wavenumber Check + - filter: RejectList + filter variables: + - name: brightnessTemperature + channels: 1972, 1973, 1974, 1975, 1976, 1977, 1978, 1979, 1980, 1981, + 1982, 1983, 1984, 1985, 1986, 1987, 2119, 2140, 2143, 2147, + 2153, 2158, 2161, 2168, 2171, 2175, 2182 + where: + - variable: + name: MetaData/solarZenithAngle + maxvalue: 88.9999 + - variable: + name: GeoVaLs/water_area_fraction + minvalue: 1.0e-12 + action: + name: reject + + # Step 7: Observation Error Inflation based on Wavenumber + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorWavenumIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 8: Observation Range Sanity Check + # Valid range: (50, 550) + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + minvalue: 50.00001 + maxvalue: 549.99999 + action: + name: reject + + # Step 9: Topography Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorTopoRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + sensor: *{{observation_from_jcb}}_sensor_id + + # Step 10: Transmittance Top Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorTransmitTopRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 11: Cloud Detection Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/CloudDetectMinResidualIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: &{{observation_from_jcb}}_active_channels [{{ get_satellite_variable(observation_from_jcb, "active") }}] + use_flag_clddet: [{{ get_satellite_variable(observation_from_jcb, "icld_det") }}] + obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] + error parameter vector: *{{observation_from_jcb}}_oberr + maxvalue: 1.0e-12 + action: + name: reject + + # Step 12: Surface Temperature Jacobian Check over Land + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + test variables: + - name: ObsDiag/brightness_temperature_jacobian_skin_temperature_at_surface + channels: *{{observation_from_jcb}}_simulated_channels + maxvalue: 0.2 + + # Step 13: NSST Retrieval Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/NearSSTRetCheckIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: *{{observation_from_jcb}}_active_channels + obserr_demisf: [0.01, 0.02, 0.03, 0.02, 0.03] + obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] + maxvalue: 1.0e-12 + action: + name: reject + + # Step 14: Surface Jacobians Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorSurfJacobianRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_demisf: [0.01, 0.02, 0.03, 0.02, 0.03] + obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] + sensor: *{{observation_from_jcb}}_sensor_id + + # Step 15: Gross check + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + function absolute threshold: + - name: ObsFunction/ObsErrorBoundIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_latitude: + name: ObsFunction/ObsErrorFactorLatRad + options: + latitude_parameters: [25.0, 0.5, 0.04, 1.0] + obserr_bound_transmittop: + name: ObsFunction/ObsErrorFactorTransmitTopRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_max: [{{ get_satellite_variable(observation_from_jcb, "ermax") }}] + error parameter vector: *{{observation_from_jcb}}_oberr + action: + name: reject + + # Step 16: Useflag Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/ChannelUseflagCheckRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: *{{observation_from_jcb}}_active_channels + minvalue: 1.0e-12 + action: + name: reject # GeoVaLs for Driving Observation Operators (testing mode) # -------------------------------------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 index ab41e565d..5454945a4 100644 --- a/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 @@ -14,7 +14,7 @@ obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 - observed variables: [radiance] + observed variables: [spectralRadiance] simulated variables: [brightnessTemperature] derived variables: [brightnessTemperature] channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} @@ -36,6 +36,8 @@ obs bias: input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" + variables without bc: [brightnessTemperature] + channels: {{ get_satellite_variable(observation_from_jcb, "not_biascorrtd") }} variational bc: predictors: - name: constant @@ -78,7 +80,7 @@ name: ObsFunction/LinearCombination options: variables: - - name: ObsValue/radiance + - name: ObsValue/spectralRadiance channels: *{{observation_from_jcb}}_simulated_channels coefs: [25.0] intercept: 64475.0 @@ -88,7 +90,7 @@ - filter: Variable Transforms Transform: SatBrightnessTempFromRad transform from: - name: ObsValue/radiance + name: ObsValue/spectralRadiance channels: *{{observation_from_jcb}}_simulated_channels spectral variable: name: MetaData/sensorCentralWavenumber @@ -118,7 +120,19 @@ action: name: reduce obs space - # Step 5: Data Thinning + # Step 5: Create cloudFree variable (for thinning) + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/cloudFree + type: int + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsValue/cloudFree + coefs: [100] + + # Step 6: Create cloudFree variable - filter: Gaussian Thinning filter variables: - name: brightnessTemperature @@ -126,14 +140,14 @@ horizontal_mesh: 145 use_reduced_horizontal_grid: true distance_norm: geodesic - priority_variable: MetaData/fractionOfClearPixelsInFOV + priority_variable: DerivedMetaData/cloudFree action: name: reduce obs space # Observation Post Filters (QC) # ----------------------------- obs post filters: - # Step 6: Wavenumber Check + # Step 7: Wavenumber Check - filter: Perform Action filter variables: - name: brightnessTemperature @@ -152,7 +166,7 @@ action: name: reject - # Step 7: Observation Error Inflation based on Wavenumber + # Step 8: Observation Error Inflation based on Wavenumber - filter: Perform Action filter variables: - name: brightnessTemperature @@ -165,7 +179,7 @@ options: channels: *{{observation_from_jcb}}_simulated_channels - # Step 8: Observation Range Sanity Check + # Step 9: Observation Range Sanity Check # Valid range: (50, 550) - filter: Bounds Check filter variables: @@ -176,7 +190,7 @@ action: name: reject - # Step 9: Topography Check + # Step 10: Topography Check - filter: Perform Action filter variables: - name: brightnessTemperature @@ -190,7 +204,7 @@ channels: *{{observation_from_jcb}}_simulated_channels sensor: *{{observation_from_jcb}}_sensor_id - # Step 10: Transmittance Top Check + # Step 11: Transmittance Top Check - filter: Perform Action filter variables: - name: brightnessTemperature @@ -203,7 +217,7 @@ options: channels: *{{observation_from_jcb}}_simulated_channels - # Step 11: Cloud Detection Check + # Step 12: Cloud Detection Check - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -221,7 +235,7 @@ action: name: reject - # Step 12: NSST Retrieval Check + # Step 13: NSST Retrieval Check - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -238,7 +252,7 @@ action: name: reject - # Step 13: Surface Jacobians Check + # Step 14: Surface Jacobians Check - filter: Perform Action filter variables: - name: brightnessTemperature @@ -254,7 +268,7 @@ obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] sensor: *{{observation_from_jcb}}_sensor_id - # Step 14: Gross check + # Step 15: Gross check - filter: Background Check filter variables: - name: brightnessTemperature @@ -278,7 +292,7 @@ action: name: reject - # Step 15: Useflag Check + # Step 16: Useflag Check - filter: Bounds Check filter variables: - name: brightnessTemperature diff --git a/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 index 1490345eb..733b71658 100644 --- a/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 @@ -14,7 +14,7 @@ obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 - observed variables: [radiance] + observed variables: [spectralRadiance] simulated variables: [brightnessTemperature] derived variables: [brightnessTemperature] channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} @@ -36,6 +36,8 @@ obs bias: input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" + variables without bc: [brightnessTemperature] + channels: {{ get_satellite_variable(observation_from_jcb, "not_biascorrtd") }} variational bc: predictors: - name: constant @@ -78,7 +80,7 @@ name: ObsFunction/LinearCombination options: variables: - - name: ObsValue/radiance + - name: ObsValue/spectralRadiance channels: *{{observation_from_jcb}}_simulated_channels coefs: [25.0] intercept: 64475.0 @@ -88,7 +90,7 @@ - filter: Variable Transforms Transform: SatBrightnessTempFromRad transform from: - name: ObsValue/radiance + name: ObsValue/spectralRadiance channels: *{{observation_from_jcb}}_simulated_channels spectral variable: name: MetaData/sensorCentralWavenumber @@ -118,7 +120,19 @@ action: name: reduce obs space - # Step 5: Data Thinning + # Step 5: Create cloudFree variable (for thinning) + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/cloudFree + type: int + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsValue/cloudFree + coefs: [100] + + # Step 6: Create cloudFree variable - filter: Gaussian Thinning filter variables: - name: brightnessTemperature @@ -126,14 +140,14 @@ horizontal_mesh: 145 use_reduced_horizontal_grid: true distance_norm: geodesic - priority_variable: MetaData/fractionOfClearPixelsInFOV + priority_variable: DerivedMetaData/cloudFree action: name: reduce obs space # Observation Post Filters (QC) # ----------------------------- obs post filters: - # Step 6: Wavenumber Check + # Step 7: Wavenumber Check - filter: Perform Action filter variables: - name: brightnessTemperature @@ -152,7 +166,7 @@ action: name: reject - # Step 7: Observation Error Inflation based on Wavenumber + # Step 8: Observation Error Inflation based on Wavenumber - filter: Perform Action filter variables: - name: brightnessTemperature @@ -165,7 +179,7 @@ options: channels: *{{observation_from_jcb}}_simulated_channels - # Step 8: Observation Range Sanity Check + # Step 9: Observation Range Sanity Check # Valid range: (50, 550) - filter: Bounds Check filter variables: @@ -176,7 +190,7 @@ action: name: reject - # Step 9: Topography Check + # Step 10: Topography Check - filter: Perform Action filter variables: - name: brightnessTemperature @@ -190,7 +204,7 @@ channels: *{{observation_from_jcb}}_simulated_channels sensor: *{{observation_from_jcb}}_sensor_id - # Step 10: Transmittance Top Check + # Step 11: Transmittance Top Check - filter: Perform Action filter variables: - name: brightnessTemperature @@ -203,7 +217,7 @@ options: channels: *{{observation_from_jcb}}_simulated_channels - # Step 11: Cloud Detection Check + # Step 12: Cloud Detection Check - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -221,7 +235,7 @@ action: name: reject - # Step 12: NSST Retrieval Check + # Step 13: NSST Retrieval Check - filter: Bounds Check filter variables: - name: brightnessTemperature @@ -238,7 +252,7 @@ action: name: reject - # Step 13: Surface Jacobians Check + # Step 14: Surface Jacobians Check - filter: Perform Action filter variables: - name: brightnessTemperature @@ -254,7 +268,7 @@ obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] sensor: *{{observation_from_jcb}}_sensor_id - # Step 14: Gross check + # Step 15: Gross check - filter: Background Check filter variables: - name: brightnessTemperature @@ -278,7 +292,7 @@ action: name: reject - # Step 15: Useflag Check + # Step 16: Useflag Check - filter: Bounds Check filter variables: - name: brightnessTemperature diff --git a/parm/jcb-gdas/observations/atmosphere/iasi_metop-c.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/iasi_metop-c.yaml.j2 new file mode 100644 index 000000000..9565aac22 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/iasi_metop-c.yaml.j2 @@ -0,0 +1,317 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: IASI METOP-C + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + observed variables: [spectralRadiance] + simulated variables: [brightnessTemperature] + derived variables: [brightnessTemperature] + channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} + + # Observation Operator + # -------------------- + obs operator: + name: CRTM + Absorbers: [H2O, O3, CO2] + obs options: + Sensor_ID: &{{observation_from_jcb}}_sensor_id iasi_metop-c + EndianType: little_endian + CoefficientPath: "{{crtm_coefficient_path}}" + linear obs operator: + Absorbers: [H2O, O3] + + # Observation Bias Correction (VarBC) + # ----------------------------------- + obs bias: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" + variables without bc: [brightnessTemperature] + channels: {{ get_satellite_variable(observation_from_jcb, "not_biascorrtd") }} + variational bc: + predictors: + - name: constant + - name: lapseRate + order: 2 + tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" + - name: lapseRate + tlapse: *{{observation_from_jcb}}_tlapse + - name: emissivityJacobian + - name: sensorScanAngle + order: 4 + - name: sensorScanAngle + order: 3 + - name: sensorScanAngle + order: 2 + - name: sensorScanAngle + covariance: + minimal required obs number: 20 + variance range: [1.0e-6, 10.0] + step size: 1.0e-4 + largest analysis variance: 10000.0 + prior: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiascovin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovin_suffix}}" + inflation: + ratio: 1.1 + ratio for small dataset: 2.0 + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" + + # Observation Pre Filters (QC) + # ---------------------------- + obs pre filters: + # Step 1: Create Derived Variables + # Assign channel wavenumbers in m-1 + - filter: Variable Assignment + assignments: + - name: MetaData/sensorCentralWavenumber + type: float + channels: *{{observation_from_jcb}}_simulated_channels + function: + name: ObsFunction/LinearCombination + options: + variables: + - name: ObsValue/spectralRadiance + channels: *{{observation_from_jcb}}_simulated_channels + coefs: [25.0] + intercept: 64475.0 + use channel numbers: true + + # Step 2: Transform radiance to brightness temperature + - filter: Variable Transforms + Transform: SatBrightnessTempFromRad + transform from: + name: ObsValue/spectralRadiance + channels: *{{observation_from_jcb}}_simulated_channels + spectral variable: + name: MetaData/sensorCentralWavenumber + channels: *{{observation_from_jcb}}_simulated_channels + radiance units: wavenumber + planck1: 1.191042953e-16 + planck2: 1.4387774e-2 + + # Step 3: Assign Observation Error + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: assign error + error parameter vector: &{{observation_from_jcb}}_oberr [{{ get_satellite_variable(observation_from_jcb, "error") }}] + + # Step 4: Remove Observations from the Edge of the Scan + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: MetaData/sensorScanPosition + is_in: 5-56 + action: + name: reduce obs space + + # Step 5: Create cloudFree variable (for thinning) + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/cloudFree + type: int + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsValue/cloudFree + coefs: [100] + + # Step 6: Create cloudFree variable + - filter: Gaussian Thinning + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + horizontal_mesh: 145 + use_reduced_horizontal_grid: true + distance_norm: geodesic + priority_variable: DerivedMetaData/cloudFree + action: + name: reduce obs space + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Step 7: Wavenumber Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: 7024, 7027, 7029, 7032, 7038, 7043, 7046, 7049, 7069, 7072, 7076, + 7081, 7084, 7089, 7099, 7209, 7222, 7231, 7235, 7247, 7267, 7269, 7284, 7389, + 7419, 7423, 7424, 7426, 7428, 7431, 7436, 7444, 7475, 7549, 7584, 7665, 7666, + 7831, 7836, 7853, 7865, 7885, 7888, 7912, 7950, 7972, 7980, 7995, 8007, 8015, + 8055, 8078 + where: + - variable: + name: MetaData/solarZenithAngle + maxvalue: 88.9999 + - variable: + name: GeoVaLs/water_area_fraction + minvalue: 1.0e-12 + action: + name: reject + + # Step 8: Observation Error Inflation based on Wavenumber + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorWavenumIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 9: Observation Range Sanity Check + # Valid range: (50, 550) + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + minvalue: 50.00001 + maxvalue: 549.99999 + action: + name: reject + + # Step 10: Topography Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorTopoRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + sensor: *{{observation_from_jcb}}_sensor_id + + # Step 11: Transmittance Top Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorTransmitTopRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + + # Step 12: Cloud Detection Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/CloudDetectMinResidualIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: &{{observation_from_jcb}}_active_channels [{{ get_satellite_variable(observation_from_jcb, "active") }}] + use_flag_clddet: [{{ get_satellite_variable(observation_from_jcb, "icld_det") }}] + obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] + error parameter vector: *{{observation_from_jcb}}_oberr + maxvalue: 1.0e-12 + action: + name: reject + + # Step 13: NSST Retrieval Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/NearSSTRetCheckIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: *{{observation_from_jcb}}_active_channels + obserr_demisf: [0.01, 0.02, 0.03, 0.02, 0.03] + obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] + maxvalue: 1.0e-12 + action: + name: reject + + # Step 14: Surface Jacobians Check + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorSurfJacobianRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_demisf: [0.01, 0.02, 0.03, 0.02, 0.03] + obserr_dtempf: [0.50, 2.00, 4.00, 2.00, 4.00] + sensor: *{{observation_from_jcb}}_sensor_id + + # Step 15: Gross check + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + function absolute threshold: + - name: ObsFunction/ObsErrorBoundIR + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_latitude: + name: ObsFunction/ObsErrorFactorLatRad + options: + latitude_parameters: [25.0, 0.5, 0.04, 1.0] + obserr_bound_transmittop: + name: ObsFunction/ObsErrorFactorTransmitTopRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + obserr_bound_max: [{{ get_satellite_variable(observation_from_jcb, "ermax") }}] + error parameter vector: *{{observation_from_jcb}}_oberr + action: + name: reject + + # Step 16: Useflag Check + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: ObsFunction/ChannelUseflagCheckRad + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + use_flag: *{{observation_from_jcb}}_active_channels + minvalue: 1.0e-12 + action: + name: reject + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/omi_aura.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ozone.omi_aura.yaml.j2 similarity index 71% rename from parm/jcb-gdas/observations/atmosphere/omi_aura.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/ozone.omi_aura.yaml.j2 index d8b6d1be0..d04c894d5 100644 --- a/parm/jcb-gdas/observations/atmosphere/omi_aura.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ozone.omi_aura.yaml.j2 @@ -44,35 +44,8 @@ name: assign error error parameter: 6.0 - - filter: Create Diagnostic Flags - filter variables: - - name: ozoneTotal - flags: - - name: ObsValueSanityCheck - initial value: false - force reinitialization: false - - name: RowAnomaly - initial value: false - force reinitialization: false - - name: BadScan - initial value: false - force reinitialization: false - - name: Thinning - initial value: false - force reinitialization: false - - name: RetrievalQualityCodeFlag - initial value: false - force reinitialization: false - - name: RetrievalQualityAlgorithmFlag - initial value: false - force reinitialization: false - - name: GrossCheck - initial value: false - force reinitialization: false - # Observation Prior Filters (QC) # ------------------------------ - obs prior filters: # GSI read routine QC # range sanity check - filter: Bounds Check @@ -80,10 +53,8 @@ - name: ozoneTotal minvalue: 0 maxvalue: 10000 - actions: - - name: set - flag: ObsValueSanityCheck - - name: reject + action: + name: reject # Do not use the data if row anomaly (bit 10)is 1 - filter: RejectList @@ -93,10 +64,8 @@ - variable: name: MetaData/totalOzoneQualityFlag any_bit_set_of: 9 - actions: - - name: set - flag: RowAnomaly - - name: reject + action: + name: reject # Scan position check: reject scan position >= 25 - filter: RejectList @@ -106,10 +75,8 @@ - variable: name: MetaData/sensorScanPosition minvalue: 25 - actions: - - name: set - flag: BadScan - - name: reject + action: + name: reject # Accept total_ozone_error_flag values of 0 and 1, but not any others. - filter: RejectList @@ -119,10 +86,8 @@ - variable: name: MetaData/totalOzoneQualityCode is_not_in: 0, 1 - actions: - - name: set - flag: RetrievalQualityCodeFlag - - name: reject + action: + name: reject # Use data with best ozone algorighm - filter: RejectList @@ -132,20 +97,16 @@ - variable: name: MetaData/bestOzoneAlgorithmFlag is_in: 3, 13 - actions: - - name: set - flag: RetrievalQualityAlgorithmFlag - - name: reject + action: + name: reject # Data Thinning - filter: Gaussian Thinning horizontal_mesh: 150 use_reduced_horizontal_grid: true distance_norm: geodesic - actions: - - name: set - flag: Thinning - - name: reject + action: + name: reduce obs space # Observation Post Filters (QC) # ----------------------------- @@ -157,10 +118,8 @@ - name: ozoneTotal threshold: 10.0 absolute threshold: 300.0 - actions: - - name: set - flag: GrossCheck - - name: reject + action: + name: reject # End of Filters diff --git a/parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_n20.yaml.j2 new file mode 100644 index 000000000..6c3033fef --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_n20.yaml.j2 @@ -0,0 +1,332 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: ompsnp_n20 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsgrouping: + group variables: ["latitude"] + sort variable: "pressure" + sort order: "ascending" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [ozoneLayer] + + #obs operator: + # name: AtmVertInterpLay + # geovals: [mole_fraction_of_ozone_in_air] + # coefficients: [0.007886131] # convert from ppmv to DU + # nlevels: [22] + + # Observation Operator + # -------------------- + obs operator: + name: ColumnRetrieval + nlayers_retrieval: 1 + tracer variables: [mole_fraction_of_ozone_in_air] + isApriori: false + isAveragingKernel: false + totalNoVertice: false + stretchVertices: none #options: top, bottom, topbottom, none + # model units coeff: 2.240013904035E-3 # this number to match the gsihofx values + model units coeff: 2.241398632746E-3 # this number to match the gsihofx values (use this) + # the actual scientific conversion factor is + # 2.1415E-3 kg[O3]/m-2 to DU + # so the name of the geovals + # is also likely wrong, as it apprears to be a mass + # fraction given the conversion factor needed + + # Observation Pre Filters (QC) + # ---------------------------- + obs pre filters: + # Observation error assignment + - filter: Perform Action + filter variables: + - name: ozoneLayer + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [0.001, 10.1325, 16.00935, 25.43258, 40.32735, 63.93607, 101.325, + 160.0935, 254.3257, 403.2735, 639.3608, 1013.25, 1600.935, 2543.258, 4032.735, + 6393.607, 10132.5, 16009.35, 25432.57, 40327.35, 63936.07, 101325] + # errors: [7.7236, 0.020, 0.020, 0.025, 0.080, 0.150, 0.056, 0.125, 0.200, + # 0.299, 0.587, 0.864, 1.547, 2.718, 3.893, 4.353, 3.971, 4.407, 4.428, + # 3.312, 2.198, 2.285] + errors: [7.7236, 0.020, 0.020, 0.025, 0.040, 0.080, 0.156, 0.245, 0.510, + 1.098, 3.917, 6.124, 6.347, 5.798, 6.843, 9.253,10.091,10.967, 8.478, + 5.572, 2.638, 3.525] # operational from gfs.v16.3.9 (late 2023) + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Do not assimilation where pressure is zero + # Zero pressure indicates the data is total column ozone + - filter: RejectList + filter variables: + - name: ozoneLayer + where: + - variable: + name: MetaData/pressure + maxvalue: 0.0001 + + # Sanity check on observaton values + - filter: Bounds Check + filter variables: + - name: ozoneLayer + minvalue: 0 + maxvalue: 1000 + action: + name: reject + + # Total Ozone Quality Check (keeps 0, 2) + # 0 indentifies good data + # 2 identifies good data with a solar zenith angle > 84 degrees + - filter: RejectList + filter variables: + - name: ozoneLayer + where: + - variable: + name: MetaData/totalOzoneQuality + is_not_in: 0, 2 + + # Profile Ozone Quality Check (keeps 0, 1, 7) + # 0 : good data + # 1 : good data with a solar zenith angle > 84 degrees + # 7 : profile for which stray light correction applied + - filter: RejectList + filter variables: + - name: ozoneLayer + where: + - variable: + name: MetaData/profileOzoneQuality + is_not_in: 0, 1, 7 + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Gross error check + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 120 + action: + name: reject + where: + - variable: + name: MetaData/pressure + maxvalue: 0.001 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 30 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 30000.0 + maxvalue: 110000.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 40 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 20000.0 + maxvalue: 30000.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 44.42 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 10100.0 + maxvalue: 20000.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 57.52 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 6400.0 + maxvalue: 10100.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 69.4 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 4000.0 + maxvalue: 6400.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 70 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 2600.0 + maxvalue: 4000.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 62.73 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 1600.0 + maxvalue: 2600.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 50.52 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 1100.0 + maxvalue: 1600.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 35.9 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 700.0 + maxvalue: 1100.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 26.41 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 400.0 + maxvalue: 700.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 20.51 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 300.0 + maxvalue: 400.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 12.82 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 200.0 + maxvalue: 300.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 10 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 70.0 + maxvalue: 200.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 5 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 40.0 + maxvalue: 70.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 2 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 30.0 + maxvalue: 40.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 1 + action: + name: reject + where: + - variable: + name: MetaData/pressure + maxvalue: 30.0 + + # End of Filters + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/ompsnp_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_npp.yaml.j2 similarity index 95% rename from parm/jcb-gdas/observations/atmosphere/ompsnp_npp.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_npp.yaml.j2 index 807eb9478..161ab9365 100644 --- a/parm/jcb-gdas/observations/atmosphere/ompsnp_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_npp.yaml.j2 @@ -61,10 +61,12 @@ xvals: [0.001, 10.1325, 16.00935, 25.43258, 40.32735, 63.93607, 101.325, 160.0935, 254.3257, 403.2735, 639.3608, 1013.25, 1600.935, 2543.258, 4032.735, 6393.607, 10132.5, 16009.35, 25432.57, 40327.35, 63936.07, 101325] - errors: [7.7236, 0.020, 0.020, 0.025, 0.080, 0.150, 0.056, 0.125, 0.200, - 0.299, 0.587, 0.864, 1.547, 2.718, 3.893, 4.353, 3.971, 4.407, 4.428, - 3.312, 2.198, 2.285] - # errors: [7.7236, 0.020, 0.020, 0.025, 0.040, 0.080, 0.156, 0.245, 0.510, 1.098, 3.917, 6.124, 6.347, 5.798, 6.843, 9.253,10.091,10.967, 8.478, 5.572, 2.638, 3.525] # operational from gfs.v16.3.9 (late 2023) + # errors: [7.7236, 0.020, 0.020, 0.025, 0.080, 0.150, 0.056, 0.125, 0.200, + # 0.299, 0.587, 0.864, 1.547, 2.718, 3.893, 4.353, 3.971, 4.407, 4.428, + # 3.312, 2.198, 2.285] + errors: [7.7236, 0.020, 0.020, 0.025, 0.040, 0.080, 0.156, 0.245, 0.510, + 1.098, 3.917, 6.124, 6.347, 5.798, 6.843, 9.253,10.091,10.967, 8.478, + 5.572, 2.638, 3.525] # operational from gfs.v16.3.9 (late 2023) # Observation Prior Filters (QC) # ------------------------------ diff --git a/parm/jcb-gdas/observations/atmosphere/ozone.ompstc_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ozone.ompstc_n20.yaml.j2 new file mode 100644 index 000000000..2d369a30e --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/ozone.ompstc_n20.yaml.j2 @@ -0,0 +1,133 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: ompstc_n20 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [ozoneTotal] + + #obs operator: + # name: AtmVertInterpLay + # geovals: [mole_fraction_of_ozone_in_air] + # coefficients: [0.007886131] # convert from ppmv to DU + # nlevels: [1] + + # Observation Operator + # -------------------- + obs operator: + name: ColumnRetrieval + nlayers_retrieval: 1 + tracer variables: [mole_fraction_of_ozone_in_air] + isApriori: false + isAveragingKernel: false + totalNoVertice: true + stretchVertices: topbottom # options: top, bottom, topbottom, none + model units coeff: 2.241398632746E-3 + + # Observation Pre Filters (QC) + # ---------------------------- + obs pre filters: + # assign observation error + - filter: Perform Action + filter variables: + - name: ozoneTotal + action: + name: assign error + error parameter: 6.0 + + # GSI read routine QC + # range sanity check + - filter: Bounds Check + filter variables: + - name: ozoneTotal + minvalue: 0 + maxvalue: 1000 + action: + name: reject + + #- filter: Gaussian Thinning + # horizontal_mesh: 150 + # use_reduced_horizontal_grid: true + # distance_norm: geodesic + # action: + # name: reject + + # Accept total_ozone_error_flag values of 0 and 1, but not any others. + - filter: RejectList + filter variables: + - name: ozoneTotal + where: + - variable: + name: MetaData/totalOzoneQualityCode + is_not_in: 0, 1 + + - filter: RejectList + filter variables: + - name: ozoneTotal + where: + - variable: + name: MetaData/bestOzoneAlgorithmFlag + is_in: 3, 13 + + # GSI setup routine QC + - filter: RejectList + filter variables: + - name: ozoneTotal + where: + - variable: + name: MetaData/sensorScanPosition + is_in: 1, 2, 3, 4, 35 + - variable: + name: MetaData/latitude + minvalue: 50.0 + + - filter: RejectList + filter variables: + - name: ozoneTotal + where: + - variable: + name: MetaData/sensorScanPosition + is_in: 1, 2, 3, 4, 35 + - variable: + name: MetaData/latitude + maxvalue: -50.0 + + - filter: Gaussian Thinning + horizontal_mesh: 150 + use_reduced_horizontal_grid: true + distance_norm: geodesic + action: + name: reduce obs space + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + - filter: Background Check + filter variables: + - name: ozoneTotal + threshold: 10.0 + absolute threshold: 300.0 + action: + name: reject + + # End of Filters + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/ompstc_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ozone.ompstc_npp.yaml.j2 similarity index 96% rename from parm/jcb-gdas/observations/atmosphere/ompstc_npp.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/ozone.ompstc_npp.yaml.j2 index c6e388471..47596cfe4 100644 --- a/parm/jcb-gdas/observations/atmosphere/ompstc_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ozone.ompstc_npp.yaml.j2 @@ -37,6 +37,7 @@ # Observation Pre Filters (QC) # ---------------------------- obs pre filters: + # assign observation error - filter: Perform Action filter variables: - name: ozoneTotal @@ -44,9 +45,6 @@ name: assign error error parameter: 6.0 - # Observation Prior Filters (QC) - # ------------------------------ - obs prior filters: # GSI read routine QC # range sanity check - filter: Bounds Check @@ -109,7 +107,7 @@ use_reduced_horizontal_grid: true distance_norm: geodesic action: - name: reject + name: reduce obs space # Observation Post Filters (QC) # ----------------------------- diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 index 94b0458cc..163b29c22 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 @@ -80,6 +80,7 @@ name: MetaData/pressure xvals: [110000., 0.] #Pressure (Pa) errors: [1000000000., 1000000000.] + # Type 245 (GOES LWIR): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 - filter: Perform Action filter variables: @@ -102,6 +103,7 @@ errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14.] + # Type 246 (GOES cloud-top WV): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 - filter: Perform Action filter variables: @@ -124,6 +126,7 @@ errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14.] + # Type 247 (GOES clear-sky WV): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 - filter: Perform Action filter variables: @@ -146,6 +149,7 @@ errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14.] + # Type 251 (GOES VIS): Assigned all dummy values - filter: Perform Action filter variables: @@ -163,6 +167,7 @@ name: MetaData/pressure xvals: [110000., 0.] #Pressure (Pa) errors: [1000000000., 1000000000.] + # sanity-check criteria # Observation Range Sanity Check # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-17.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-17.yaml.j2 index b0a41796f..86a799712 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-17.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-17.yaml.j2 @@ -80,6 +80,7 @@ name: MetaData/pressure xvals: [110000., 0.] #Pressure (Pa) errors: [1000000000., 1000000000.] + # Type 245 (GOES LWIR): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 - filter: Perform Action filter variables: @@ -102,6 +103,7 @@ errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14.] + # Type 246 (GOES cloud-top WV): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 - filter: Perform Action filter variables: @@ -124,6 +126,7 @@ errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14.] + # Type 247 (GOES clear-sky WV): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 - filter: Perform Action filter variables: @@ -146,6 +149,7 @@ errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14.] + # Type 251 (GOES VIS): Assigned all dummy values - filter: Perform Action filter variables: @@ -163,6 +167,7 @@ name: MetaData/pressure xvals: [110000., 0.] #Pressure (Pa) errors: [1000000000., 1000000000.] + # sanity-check criteria # Observation Range Sanity Check # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested @@ -445,7 +450,7 @@ - variable: name: ObsType/windEastward is_in: 247 - reference: GeoVaLs/surface_pressure + reference: GeoVaLs/air_pressure_at_surface value: MetaData/pressure maxvalue: -11000. # within 110 hPa above surface pressure, negative p-diff action: diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-18.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-18.yaml.j2 index f1e61fbed..b1a7a6ed7 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-18.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-18.yaml.j2 @@ -80,6 +80,7 @@ name: MetaData/pressure xvals: [110000., 0.] #Pressure (Pa) errors: [1000000000., 1000000000.] + # Type 245 (GOES LWIR): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 - filter: Perform Action filter variables: @@ -102,6 +103,7 @@ errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14.] + # Type 246 (GOES cloud-top WV): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 - filter: Perform Action filter variables: @@ -124,6 +126,7 @@ errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14.] + # Type 247 (GOES clear-sky WV): I am assuming these are halved relative to prepobs_errtable.global, based on read_satwnd.f90: L1410–1416 - filter: Perform Action filter variables: @@ -146,6 +149,7 @@ errors: [7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.8, 7.8, 8., 8., 8.2, 10., 12., 12.6, 13.2, 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14., 14.] + # Type 251 (GOES VIS): Assigned all dummy values - filter: Perform Action filter variables: @@ -163,6 +167,7 @@ name: MetaData/pressure xvals: [110000., 0.] #Pressure (Pa) errors: [1000000000., 1000000000.] + # sanity-check criteria # Observation Range Sanity Check # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested @@ -445,7 +450,7 @@ - variable: name: ObsType/windEastward is_in: 247 - reference: GeoVaLs/surface_pressure + reference: GeoVaLs/air_pressure_at_surface value: MetaData/pressure maxvalue: -11000. # within 110 hPa above surface pressure, negative p-diff action: diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h8.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h8.yaml.j2 index 538957071..424858e68 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h8.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h8.yaml.j2 @@ -28,51 +28,9 @@ linear obs operator: name: VertInterp - # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI"s thinning of AHI/Himawari-8 satwinds - # results in more JEDI satwinds in the diag file than in GSI, but far fewer JEDI satwinds assimilated than - # GSI. JEDI under-counts assimilated winds by roughly 25-40%, relative to GSI, and this under-count is not - # even including the temporal thinning which is applied in GSI but not JEDI (by this filter below). See - # GDASApp Issue #741 for details: https://github.com/NOAA-EMC/GDASApp/issues/741 - #obs pre filters: - #- filter: Gaussian Thinning - # horizontal_mesh: 200 - # vertical_mesh: 10000 - # use_reduced_horizontal_grid: true - # round_horizontal_bin_count_to_nearest: true - # partition_longitude_bins_using_mesh: true - - # Observation Prior Filters (QC) - # ------------------------------ - obs prior filters: - # Apply variable changes needed for wind scaling - # For wind observations with pressure provided - - filter: Variable Transforms - Transform: SurfaceWindScalingPressure - SkipWhenNoObs: false - - # Calculate error inflation factor for duplicate observations - #- filter: Variable Assignment - # assignments: - # - name: ObsErrorFactorDuplicateCheck/windEastward - # type: float - # function: - # name: ObsFunction/ObsErrorFactorDuplicateCheck - # options: - # use_air_pressure: true - # variable: windEastward - #- filter: Variable Assignment - # assignments: - # - name: ObsErrorFactorDuplicateCheck/windNorthward - # type: float - # function: - # name: ObsFunction/ObsErrorFactorDuplicateCheck - # options: - # use_air_pressure: true - # variable: windNorthward - - # Observation Post Filters (QC) + # Observation Pre Filters (QC) # ----------------------------- - obs post filters: + obs pre filters: # Assign the initial observation error, based on height/pressure # Hard-wiring to prepobs_errtable.global by Type # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML @@ -98,6 +56,7 @@ errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7.] + # Type 250 (Himawari AHI WV, cloud-top or clear-sky) - filter: Perform Action filter variables: @@ -120,6 +79,7 @@ errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, 5., 7., 7.3, 7.6, 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., 8.] + # Type Type 252 (Himawari AHI LWIR) - filter: Perform Action filter variables: @@ -142,6 +102,7 @@ errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7.] + # sanity-check criteria # Observation Range Sanity Check # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested @@ -204,6 +165,48 @@ action: name: reject +# # Reject Type 252 (IR) winds with a /=0 surface type (non-water surface) when latitude > 20. +# - filter: Bounds Check +# filter variables: +# - name: windEastward +# - name: windNorthward +# where: +# - variable: +# name: GeoVaLs/water_area_fraction +# maxvalue: 0.99 +# - variable: +# name: ObsType/windEastward +# is_in: 252 +# test variables: +# - name: MetaData/latitude +# maxvalue: 20. +# action: +# name: reject + + # THINNING: Himawari winds are not prioritized + - filter: Gaussian Thinning + horizontal_mesh: 100 + vertical_mesh: 10000 + where: + - variable: + name: MetaData/pressure + minvalue: 50001. + action: + name: reduce obs space + + - filter: Gaussian Thinning + horizontal_mesh: 90 + vertical_mesh: 10000 + where: + - variable: + name: MetaData/pressure + maxvalue: 50000. + action: + name: reduce obs space + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: # Reject Type 252 (IR) winds with a /=0 surface type (non-water surface) when latitude > 20. - filter: Bounds Check filter variables: @@ -222,23 +225,36 @@ action: name: reject - # THINNING: Himawari winds are not prioritized - - filter: Gaussian Thinning - horizontal_mesh: 100 - vertical_mesh: 10000 - where: - - variable: - name: MetaData/pressure - minvalue: 50001. - - filter: Gaussian Thinning - horizontal_mesh: 90 - vertical_mesh: 10000 - where: - - variable: - name: MetaData/pressure - maxvalue: 50000. - + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: false + + # Observation Post Filters (QC) + # ------------------------------ + obs post filters: # GSI setupw routine QC + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + # Reject any ob Type [240–260] when pressure greater than 950 mb. # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa - filter: Bounds Check diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h9.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h9.yaml.j2 index f69f13068..9d8c6f0c3 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h9.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h9.yaml.j2 @@ -28,51 +28,9 @@ linear obs operator: name: VertInterp - # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI"s thinning of AHI/Himawari-8 satwinds - # results in more JEDI satwinds in the diag file than in GSI, but far fewer JEDI satwinds assimilated than - # GSI. JEDI under-counts assimilated winds by roughly 25-40%, relative to GSI, and this under-count is not - # even including the temporal thinning which is applied in GSI but not JEDI (by this filter below). See - # GDASApp Issue #741 for details: https://github.com/NOAA-EMC/GDASApp/issues/741 - #obs pre filters: - #- filter: Gaussian Thinning - # horizontal_mesh: 200 - # vertical_mesh: 10000 - # use_reduced_horizontal_grid: true - # round_horizontal_bin_count_to_nearest: true - # partition_longitude_bins_using_mesh: true - - # Observation Prior Filters (QC) - # ------------------------------ - obs prior filters: - # Apply variable changes needed for wind scaling - # For wind observations with pressure provided - - filter: Variable Transforms - Transform: SurfaceWindScalingPressure - SkipWhenNoObs: false - - # Calculate error inflation factor for duplicate observations - #- filter: Variable Assignment - # assignments: - # - name: ObsErrorFactorDuplicateCheck/windEastward - # type: float - # function: - # name: ObsFunction/ObsErrorFactorDuplicateCheck - # options: - # use_air_pressure: true - # variable: windEastward - #- filter: Variable Assignment - # assignments: - # - name: ObsErrorFactorDuplicateCheck/windNorthward - # type: float - # function: - # name: ObsFunction/ObsErrorFactorDuplicateCheck - # options: - # use_air_pressure: true - # variable: windNorthward - - # Observation Post Filters (QC) + # Observation Pre Filters (QC) # ----------------------------- - obs post filters: + obs pre filters: # Assign the initial observation error, based on height/pressure # Hard-wiring to prepobs_errtable.global by Type # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML @@ -98,6 +56,7 @@ errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7.] + # Type 250 (Himawari AHI WV, cloud-top or clear-sky) - filter: Perform Action filter variables: @@ -120,6 +79,7 @@ errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, 5., 7., 7.3, 7.6, 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., 8., 8.] + # Type Type 252 (Himawari AHI LWIR) - filter: Perform Action filter variables: @@ -142,6 +102,7 @@ errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7.] + # sanity-check criteria # Observation Range Sanity Check # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested @@ -204,6 +165,48 @@ action: name: reject +# # Reject Type 252 (IR) winds with a /=0 surface type (non-water surface) when latitude > 20. +# - filter: Bounds Check +# filter variables: +# - name: windEastward +# - name: windNorthward +# where: +# - variable: +# name: GeoVaLs/water_area_fraction +# maxvalue: 0.99 +# - variable: +# name: ObsType/windEastward +# is_in: 252 +# test variables: +# - name: MetaData/latitude +# maxvalue: 20. +# action: +# name: reject + + # THINNING: Himawari winds are not prioritized + - filter: Gaussian Thinning + horizontal_mesh: 100 + vertical_mesh: 10000 + where: + - variable: + name: MetaData/pressure + minvalue: 50001. + action: + name: reduce obs space + + - filter: Gaussian Thinning + horizontal_mesh: 90 + vertical_mesh: 10000 + where: + - variable: + name: MetaData/pressure + maxvalue: 50000. + action: + name: reduce obs space + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: # Reject Type 252 (IR) winds with a /=0 surface type (non-water surface) when latitude > 20. - filter: Bounds Check filter variables: @@ -221,24 +224,37 @@ maxvalue: 20. action: name: reject + + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: false - # THINNING: Himawari winds are not prioritized - - filter: Gaussian Thinning - horizontal_mesh: 100 - vertical_mesh: 10000 - where: - - variable: - name: MetaData/pressure - minvalue: 50001. - - filter: Gaussian Thinning - horizontal_mesh: 90 - vertical_mesh: 10000 - where: - - variable: - name: MetaData/pressure - maxvalue: 50000. - + # Observation Post Filters (QC) + # ------------------------------ + obs post filters: # GSI setupw routine QC + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + # Reject any ob Type [240–260] when pressure greater than 950 mb. # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa - filter: Bounds Check diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-a.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-a.yaml.j2 new file mode 100755 index 000000000..45ce9cd58 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-a.yaml.j2 @@ -0,0 +1,289 @@ +- obs space: + name: satwind_avhrr_metop-a + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + obs operator: + name: VertInterp + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + linear obs operator: + name: VertInterp + + # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI's thinning of AHI/Himawari-8 satwinds + # results in more JEDI satwinds in the diag file than in GSI, but far fewer JEDI satwinds assimilated than + # GSI. JEDI under-counts assimilated winds by roughly 25-40%, relative to GSI, and this under-count is not + # even including the temporal thinning which is applied in GSI but not JEDI (by this filter below). See + # GDASApp Issue #741 for details: https://github.com/NOAA-EMC/GDASApp/issues/741 + #obs pre filters: + #- filter: Gaussian Thinning + # horizontal_mesh: 200 + # vertical_mesh: 10000 + # use_reduced_horizontal_grid: true + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: False + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + + obs post filters: + # Assign the initial observation error, based on height/pressure + # Hard-wiring to prepobs_errtable.global by Type + # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML + + # Type 244 (AVHRR LWIR) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 244 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000.,105000.,100000.,95000.,90000.,85000.,80000.,75000.,70000.,65000.,60000.,55000.,50000.,45000.,40000.,35000.,30000.,25000.,20000.,15000.,10000.,7500.,5000.,4000.,3000.,2000.,1000.,500.,400.,300.,200.,100.,0.] #Pressure (Pa) + errors: [3.8,3.8,3.8,3.8,3.8,3.8,3.8,3.8,3.9,3.9,4.,4.,4.1,5.,6.,6.3,6.6,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.] + + # sanity-check criteria + # Observation Range Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130. + maxvalue: 130. + action: + name: reject + + # Velocity Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130. + action: + name: reject + + # Reject any observation with a /=0 surface type (non-water surface) within + # 200 hPa of the surface pressure (as part of the LNVD check). + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + reference: GeoVaLs/air_pressure_at_surface + value: MetaData/pressure + maxvalue: -20000. # within 200 hPa above surface pressure, negative p-diff + action: + name: reject + + # LNVD check + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/SatWindsLNVDCheck + maxvalue: 3. + action: + name: reject + + # GSI setupw routine QC + # Reject any ob Type [240–260] when pressure greater than 950 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240-260 + test variables: + - name: MetaData/pressure + maxvalue: 95001. + action: + name: reject + + # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation + # Turn this check off for now. + # Need to check if troposphere pressure was implemented correctly in fv3-jed + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + reference: GeoVaLs/tropopause_pressure + value: MetaData/pressure + minvalue: -5000. # 50 hPa above tropopause level, negative p-diff + action: + name: reject + + # All satwinds must adjust errors based on ObsErrorFactorPressureCheck + # prior to the SPDB check (i.e. the gross-error check). The gross-error + # check uses the adjusted errors for error-bound tightening and rejection, + # so this check has to come first. This check will inflate errors for obs + # that are too close to either the model top or bottom. + # Notes (eliu): GMAO added a required parameter: adjusted_error_name. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + surface_obs: false + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All satwinds subject to a gross-error check that contains significant + # modifiers for satwinds with a negative speed-bias. ALL wind gross-error + # checks are currently being done by the SatWindsSPDBCheck. + # CLEARED + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [ 2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [ 2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don't want to inflate errors for duplication if one + # of the duplicates should be rejected. + # Notes (eliu): ObsErrorFactorDuplicateCheck obsfunction requires PreUseFlag (usage parameter from read_satwnd.f90). + # : Turn off duplicate check for now. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + + # We are extending this to an additional filter that inflates final ob-errors across-the-board by + # 1/0.8 = 1.25. This is caused by the GSI value of nvqc being set to .true. in the global operational + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this nvqc functionality were to be switched off (i.e. if variational qc were to be turned off), + # you would want to remove this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + + # End of Filters diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-b.yaml.j2 new file mode 100755 index 000000000..20d12c051 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-b.yaml.j2 @@ -0,0 +1,289 @@ +- obs space: + name: satwind_avhrr_metop-b + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + obs operator: + name: VertInterp + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + linear obs operator: + name: VertInterp + + # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI's thinning of AHI/Himawari-8 satwinds + # results in more JEDI satwinds in the diag file than in GSI, but far fewer JEDI satwinds assimilated than + # GSI. JEDI under-counts assimilated winds by roughly 25-40%, relative to GSI, and this under-count is not + # even including the temporal thinning which is applied in GSI but not JEDI (by this filter below). See + # GDASApp Issue #741 for details: https://github.com/NOAA-EMC/GDASApp/issues/741 + #obs pre filters: + #- filter: Gaussian Thinning + # horizontal_mesh: 200 + # vertical_mesh: 10000 + # use_reduced_horizontal_grid: true + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: False + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + + obs post filters: + # Assign the initial observation error, based on height/pressure + # Hard-wiring to prepobs_errtable.global by Type + # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML + + # Type 244 (AVHRR LWIR) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 244 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000.,105000.,100000.,95000.,90000.,85000.,80000.,75000.,70000.,65000.,60000.,55000.,50000.,45000.,40000.,35000.,30000.,25000.,20000.,15000.,10000.,7500.,5000.,4000.,3000.,2000.,1000.,500.,400.,300.,200.,100.,0.] #Pressure (Pa) + errors: [3.8,3.8,3.8,3.8,3.8,3.8,3.8,3.8,3.9,3.9,4.,4.,4.1,5.,6.,6.3,6.6,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.] + + # sanity-check criteria + # Observation Range Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130. + maxvalue: 130. + action: + name: reject + + # Velocity Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130. + action: + name: reject + + # Reject any observation with a /=0 surface type (non-water surface) within + # 200 hPa of the surface pressure (as part of the LNVD check). + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + reference: GeoVaLs/air_pressure_at_surface + value: MetaData/pressure + maxvalue: -20000. # within 200 hPa above surface pressure, negative p-diff + action: + name: reject + + # LNVD check + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/SatWindsLNVDCheck + maxvalue: 3. + action: + name: reject + + # GSI setupw routine QC + # Reject any ob Type [240–260] when pressure greater than 950 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240-260 + test variables: + - name: MetaData/pressure + maxvalue: 95001. + action: + name: reject + + # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation + # Turn this check off for now. + # Need to check if troposphere pressure was implemented correctly in fv3-jed + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + reference: GeoVaLs/tropopause_pressure + value: MetaData/pressure + minvalue: -5000. # 50 hPa above tropopause level, negative p-diff + action: + name: reject + + # All satwinds must adjust errors based on ObsErrorFactorPressureCheck + # prior to the SPDB check (i.e. the gross-error check). The gross-error + # check uses the adjusted errors for error-bound tightening and rejection, + # so this check has to come first. This check will inflate errors for obs + # that are too close to either the model top or bottom. + # Notes (eliu): GMAO added a required parameter: adjusted_error_name. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + surface_obs: false + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All satwinds subject to a gross-error check that contains significant + # modifiers for satwinds with a negative speed-bias. ALL wind gross-error + # checks are currently being done by the SatWindsSPDBCheck. + # CLEARED + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [ 2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [ 2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don't want to inflate errors for duplication if one + # of the duplicates should be rejected. + # Notes (eliu): ObsErrorFactorDuplicateCheck obsfunction requires PreUseFlag (usage parameter from read_satwnd.f90). + # : Turn off duplicate check for now. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + + # We are extending this to an additional filter that inflates final ob-errors across-the-board by + # 1/0.8 = 1.25. This is caused by the GSI value of nvqc being set to .true. in the global operational + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this nvqc functionality were to be switched off (i.e. if variational qc were to be turned off), + # you would want to remove this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + + # End of Filters diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-c.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-c.yaml.j2 new file mode 100755 index 000000000..314935408 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-c.yaml.j2 @@ -0,0 +1,289 @@ +- obs space: + name: satwind_avhrr_metop-c + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + obs operator: + name: VertInterp + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + linear obs operator: + name: VertInterp + + # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI's thinning of AHI/Himawari-8 satwinds + # results in more JEDI satwinds in the diag file than in GSI, but far fewer JEDI satwinds assimilated than + # GSI. JEDI under-counts assimilated winds by roughly 25-40%, relative to GSI, and this under-count is not + # even including the temporal thinning which is applied in GSI but not JEDI (by this filter below). See + # GDASApp Issue #741 for details: https://github.com/NOAA-EMC/GDASApp/issues/741 + #obs pre filters: + #- filter: Gaussian Thinning + # horizontal_mesh: 200 + # vertical_mesh: 10000 + # use_reduced_horizontal_grid: true + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: False + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + + obs post filters: + # Assign the initial observation error, based on height/pressure + # Hard-wiring to prepobs_errtable.global by Type + # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML + + # Type 244 (AVHRR LWIR) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 244 + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000.,105000.,100000.,95000.,90000.,85000.,80000.,75000.,70000.,65000.,60000.,55000.,50000.,45000.,40000.,35000.,30000.,25000.,20000.,15000.,10000.,7500.,5000.,4000.,3000.,2000.,1000.,500.,400.,300.,200.,100.,0.] #Pressure (Pa) + errors: [3.8,3.8,3.8,3.8,3.8,3.8,3.8,3.8,3.9,3.9,4.,4.,4.1,5.,6.,6.3,6.6,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.,7.] + + # sanity-check criteria + # Observation Range Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + minvalue: -130. + maxvalue: 130. + action: + name: reject + + # Velocity Sanity Check + # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Velocity + maxvalue: 130. + action: + name: reject + + # Reject any observation with a /=0 surface type (non-water surface) within + # 200 hPa of the surface pressure (as part of the LNVD check). + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + reference: GeoVaLs/air_pressure_at_surface + value: MetaData/pressure + maxvalue: -20000. # within 200 hPa above surface pressure, negative p-diff + action: + name: reject + + # LNVD check + # CLEARED: maxvalue is rejecting >, not >= as per a Perform Action, so threshold is unchanged + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/SatWindsLNVDCheck + maxvalue: 3. + action: + name: reject + + # GSI setupw routine QC + # Reject any ob Type [240–260] when pressure greater than 950 mb. + # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 240-260 + test variables: + - name: MetaData/pressure + maxvalue: 95001. + action: + name: reject + + # Multiple satellite platforms, reject when pressure is more than 50 mb above tropopause. + # CLEARED: minvalue is rejecting <, not <= as per a Perform Action, so threshold is unchanged + # Notes (eliu): This tropopause check reject too many obs; probably due to tropopause pressure estimation + # Turn this check off for now. + # Need to check if troposphere pressure was implemented correctly in fv3-jed + - filter: Difference Check + filter variables: + - name: windEastward + - name: windNorthward + reference: GeoVaLs/tropopause_pressure + value: MetaData/pressure + minvalue: -5000. # 50 hPa above tropopause level, negative p-diff + action: + name: reject + + # All satwinds must adjust errors based on ObsErrorFactorPressureCheck + # prior to the SPDB check (i.e. the gross-error check). The gross-error + # check uses the adjusted errors for error-bound tightening and rejection, + # so this check has to come first. This check will inflate errors for obs + # that are too close to either the model top or bottom. + # Notes (eliu): GMAO added a required parameter: adjusted_error_name. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + surface_obs: false + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 240-260 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All satwinds subject to a gross-error check that contains significant + # modifiers for satwinds with a negative speed-bias. ALL wind gross-error + # checks are currently being done by the SatWindsSPDBCheck. + # CLEARED + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [ 2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260] + cgross: [ 2.5, 2.5, 2.5, 1.5, 2.5, 1.3, 1.3, 2.5, 2.5, 2.5, 2.5, 1.3, 2.5, 1.5, 1.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5] + error_min: [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4] + error_max: [6.1, 6.1, 15.0, 15.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.1, 20.1, 20.1, 20.1, 20.1, 20.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don't want to inflate errors for duplication if one + # of the duplicates should be rejected. + # Notes (eliu): ObsErrorFactorDuplicateCheck obsfunction requires PreUseFlag (usage parameter from read_satwnd.f90). + # : Turn off duplicate check for now. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + + # We are extending this to an additional filter that inflates final ob-errors across-the-board by + # 1/0.8 = 1.25. This is caused by the GSI value of nvqc being set to .true. in the global operational + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this nvqc functionality were to be switched off (i.e. if variational qc were to be turned off), + # you would want to remove this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 240-260 + # action: + # name: inflate error + # inflation factor: 1.25 + + # End of Filters diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n15.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n15.yaml.j2 index bbe81a8c7..93483bb26 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n15.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n15.yaml.j2 @@ -119,7 +119,7 @@ - variable: name: GeoVaLs/water_area_fraction maxvalue: 0.99 - reference: GeoVaLs/surface_pressure + reference: GeoVaLs/air_pressure_at_surface value: MetaData/pressure maxvalue: -20000. # within 200 hPa above surface pressure, negative p-diff action: diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n18.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n18.yaml.j2 index fcced81ff..8fcd566d4 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n18.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n18.yaml.j2 @@ -119,7 +119,7 @@ - variable: name: GeoVaLs/water_area_fraction maxvalue: 0.99 - reference: GeoVaLs/surface_pressure + reference: GeoVaLs/air_pressure_at_surface value: MetaData/pressure maxvalue: -20000. # within 200 hPa above surface pressure, negative p-diff action: diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n19.yaml.j2 index efb84712f..357bc91e2 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n19.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n19.yaml.j2 @@ -119,7 +119,7 @@ - variable: name: GeoVaLs/water_area_fraction maxvalue: 0.99 - reference: GeoVaLs/surface_pressure + reference: GeoVaLs/air_pressure_at_surface value: MetaData/pressure maxvalue: -20000. # within 200 hPa above surface pressure, negative p-diff action: diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.modis_aqua.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.modis_aqua.yaml.j2 index d0af7524d..c539902bd 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.modis_aqua.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.modis_aqua.yaml.j2 @@ -186,7 +186,7 @@ - variable: name: GeoVaLs/water_area_fraction maxvalue: 0.99 - reference: GeoVaLs/surface_pressure + reference: GeoVaLs/air_pressure_at_surface value: MetaData/pressure maxvalue: -20000. # within 200 hPa above surface pressure, negative p-diff action: diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.modis_terra.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.modis_terra.yaml.j2 index 0f6d3f773..808a85a59 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.modis_terra.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.modis_terra.yaml.j2 @@ -198,7 +198,7 @@ - variable: name: GeoVaLs/water_area_fraction maxvalue: 0.99 - reference: GeoVaLs/surface_pressure + reference: GeoVaLs/air_pressure_at_surface value: MetaData/pressure maxvalue: -20000. # within 200 hPa above surface pressure, negative p-diff action: diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m10.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m10.yaml.j2 index c80759fac..0e0090220 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m10.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m10.yaml.j2 @@ -28,7 +28,7 @@ linear obs operator: name: VertInterp - # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI"s thinning of SEVIRI/METEOSAT-8 satwinds + # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI"s thinning of SEVIRI/METEOSAT-10 satwinds # results in more JEDI satwinds in the diag file than in GSI, but far fewer JEDI satwinds assimilated than # GSI. JEDI under-counts assimilated winds by roughly 25-40%, relative to GSI, and this under-count is not # even including the temporal thinning which is applied in GSI but not JEDI (by this filter below). See @@ -44,38 +44,9 @@ # round_horizontal_bin_count_to_nearest: true # partition_longitude_bins_using_mesh: true - # Observation Prior Filters (QC) - # ------------------------------ - obs prior filters: - # Apply variable changes needed for wind scaling - # For wind observations with pressure provided - - filter: Variable Transforms - Transform: SurfaceWindScalingPressure - SkipWhenNoObs: false - - # Calculate error inflation factor for duplicate observations - #- filter: Variable Assignment - # assignments: - # - name: ObsErrorFactorDuplicateCheck/windEastward - # type: float - # function: - # name: ObsFunction/ObsErrorFactorDuplicateCheck - # options: - # use_air_pressure: true - # variable: windEastward - #- filter: Variable Assignment - # assignments: - # - name: ObsErrorFactorDuplicateCheck/windNorthward - # type: float - # function: - # name: ObsFunction/ObsErrorFactorDuplicateCheck - # options: - # use_air_pressure: true - # variable: windNorthward - - # Observation Post Filters (QC) + # Observation Pre Filters (QC) # ----------------------------- - obs post filters: + obs pre filters: # Assign the initial observation error, based on height/pressure # Hard-wiring to prepobs_errtable.global by Type # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML @@ -103,6 +74,7 @@ errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7.] + # Type 253 (MVIRI/SEVERI LWIR) - filter: Perform Action filter variables: @@ -127,6 +99,7 @@ errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7.] + # Type 254 (MVIRI/SEVIRI WV, both cloud-top and clear-sky) - filter: Perform Action filter variables: @@ -151,6 +124,7 @@ errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4.5, 6.1, 6., 6.5, 7.3, 7.6, 7., 7.5, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7.] + # sanity-check criteria # Observation Range Sanity Check # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested @@ -200,24 +174,6 @@ action: name: reject - # Exclude data over non-water surface type where latitude > 20N for Type 253 (IRLW) --- obs tossed and not passed to setup routine - # Notes: This check was missing, so added (eliu) - # Replace land_type_index_NPOSS with water_area_fraction (eliu) - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 253 - - variable: MetaData/latitude - minvalue: 20. - test variables: - - name: GeoVaLs/water_area_fraction - minvalue: 0.99 - action: - name: reject - # GSI read routine QC (part-2) # Reject obs with qiWithoutForecast < 85. OR > 100. - filter: Bounds Check @@ -241,6 +197,9 @@ - variable: name: MetaData/pressure minvalue: 50001. + action: + name: reduce obs space + - filter: Gaussian Thinning horizontal_mesh: 90 vertical_mesh: 10000 @@ -250,10 +209,63 @@ - variable: name: MetaData/pressure maxvalue: 50000. + action: + name: reduce obs space + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: false + + # Exclude data over non-water surface type where latitude > 20N for Type 253 (IRLW) --- obs tossed and not passed to setup routine + # Notes: This check was missing, so added (eliu) + # Replace land_type_index_NPOSS with water_area_fraction (eliu) + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 253 + - variable: MetaData/latitude + minvalue: 20. + test variables: + - name: GeoVaLs/water_area_fraction + minvalue: 0.99 + action: + name: reject + + + # Observation Post Filters (QC) + # ------------------------------ + obs post filters: # GSI setupw routine QC # Reject any ob Type [240–260] when pressure greater than 950 mb. # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + - filter: Bounds Check filter variables: - name: windEastward diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m11.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m11.yaml.j2 index e8bab78e8..c6523c1c8 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m11.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m11.yaml.j2 @@ -28,7 +28,7 @@ linear obs operator: name: VertInterp - # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI"s thinning of SEVIRI/METEOSAT-8 satwinds + # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI"s thinning of SEVIRI/METEOSAT-11 satwinds # results in more JEDI satwinds in the diag file than in GSI, but far fewer JEDI satwinds assimilated than # GSI. JEDI under-counts assimilated winds by roughly 25-40%, relative to GSI, and this under-count is not # even including the temporal thinning which is applied in GSI but not JEDI (by this filter below). See @@ -44,38 +44,9 @@ # round_horizontal_bin_count_to_nearest: true # partition_longitude_bins_using_mesh: true - # Observation Prior Filters (QC) - # ------------------------------ - obs prior filters: - # Apply variable changes needed for wind scaling - # For wind observations with pressure provided - - filter: Variable Transforms - Transform: SurfaceWindScalingPressure - SkipWhenNoObs: false - - # Calculate error inflation factor for duplicate observations - #- filter: Variable Assignment - # assignments: - # - name: ObsErrorFactorDuplicateCheck/windEastward - # type: float - # function: - # name: ObsFunction/ObsErrorFactorDuplicateCheck - # options: - # use_air_pressure: true - # variable: windEastward - #- filter: Variable Assignment - # assignments: - # - name: ObsErrorFactorDuplicateCheck/windNorthward - # type: float - # function: - # name: ObsFunction/ObsErrorFactorDuplicateCheck - # options: - # use_air_pressure: true - # variable: windNorthward - - # Observation Post Filters (QC) + # Observation Pre Filters (QC) # ----------------------------- - obs post filters: + obs pre filters: # Assign the initial observation error, based on height/pressure # Hard-wiring to prepobs_errtable.global by Type # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML @@ -103,6 +74,7 @@ errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7.] + # Type 253 (MVIRI/SEVERI LWIR) - filter: Perform Action filter variables: @@ -127,6 +99,7 @@ errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7.] + # Type 254 (MVIRI/SEVIRI WV, both cloud-top and clear-sky) - filter: Perform Action filter variables: @@ -151,6 +124,7 @@ errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4.5, 6.1, 6., 6.5, 7.3, 7.6, 7., 7.5, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7.] + # sanity-check criteria # Observation Range Sanity Check # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested @@ -200,24 +174,6 @@ action: name: reject - # Exclude data over non-water surface type where latitude > 20N for Type 253 (IRLW) --- obs tossed and not passed to setup routine - # Notes: This check was missing, so added (eliu) - # Replace land_type_index_NPOSS with water_area_fraction (eliu) - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 253 - - variable: MetaData/latitude - minvalue: 20. - test variables: - - name: GeoVaLs/water_area_fraction - minvalue: 0.99 - action: - name: reject - # GSI read routine QC (part-2) # Reject obs with qiWithoutForecast < 85. OR > 100. - filter: Bounds Check @@ -230,7 +186,7 @@ maxvalue: 100. action: name: reject - + # THINNING: METEOSAT winds are prioritized by qiWithoutForecast - filter: Gaussian Thinning horizontal_mesh: 100 @@ -241,6 +197,9 @@ - variable: name: MetaData/pressure minvalue: 50001. + action: + name: reduce obs space + - filter: Gaussian Thinning horizontal_mesh: 90 vertical_mesh: 10000 @@ -250,10 +209,63 @@ - variable: name: MetaData/pressure maxvalue: 50000. + action: + name: reduce obs space + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: false + + # Exclude data over non-water surface type where latitude > 20N for Type 253 (IRLW) --- obs tossed and not passed to setup routine + # Notes: This check was missing, so added (eliu) + # Replace land_type_index_NPOSS with water_area_fraction (eliu) + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 253 + - variable: MetaData/latitude + minvalue: 20. + test variables: + - name: GeoVaLs/water_area_fraction + minvalue: 0.99 + action: + name: reject + + + # Observation Post Filters (QC) + # ------------------------------ + obs post filters: # GSI setupw routine QC # Reject any ob Type [240–260] when pressure greater than 950 mb. # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + - filter: Bounds Check filter variables: - name: windEastward diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m8.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m8.yaml.j2 index 95d33c907..b7cde2045 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m8.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m8.yaml.j2 @@ -44,38 +44,9 @@ # round_horizontal_bin_count_to_nearest: true # partition_longitude_bins_using_mesh: true - # Observation Prior Filters (QC) - # ------------------------------ - obs prior filters: - # Apply variable changes needed for wind scaling - # For wind observations with pressure provided - - filter: Variable Transforms - Transform: SurfaceWindScalingPressure - SkipWhenNoObs: false - - # Calculate error inflation factor for duplicate observations - #- filter: Variable Assignment - # assignments: - # - name: ObsErrorFactorDuplicateCheck/windEastward - # type: float - # function: - # name: ObsFunction/ObsErrorFactorDuplicateCheck - # options: - # use_air_pressure: true - # variable: windEastward - #- filter: Variable Assignment - # assignments: - # - name: ObsErrorFactorDuplicateCheck/windNorthward - # type: float - # function: - # name: ObsFunction/ObsErrorFactorDuplicateCheck - # options: - # use_air_pressure: true - # variable: windNorthward - - # Observation Post Filters (QC) + # Observation Pre Filters (QC) # ----------------------------- - obs post filters: + obs pre filters: # Assign the initial observation error, based on height/pressure # Hard-wiring to prepobs_errtable.global by Type # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML @@ -103,6 +74,7 @@ errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7.] + # Type 253 (MVIRI/SEVERI LWIR) - filter: Perform Action filter variables: @@ -127,6 +99,7 @@ errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7.] + # Type 254 (MVIRI/SEVIRI WV, both cloud-top and clear-sky) - filter: Perform Action filter variables: @@ -151,6 +124,7 @@ errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4.5, 6.1, 6., 6.5, 7.3, 7.6, 7., 7.5, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7.] + # sanity-check criteria # Observation Range Sanity Check # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested @@ -200,24 +174,6 @@ action: name: reject - # Exclude data over non-water surface type where latitude > 20N for Type 253 (IRLW) --- obs tossed and not passed to setup routine - # Notes: This check was missing, so added (eliu) - # Replace land_type_index_NPOSS with water_area_fraction (eliu) - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 253 - - variable: MetaData/latitude - minvalue: 20. - test variables: - - name: GeoVaLs/water_area_fraction - minvalue: 0.99 - action: - name: reject - # GSI read routine QC (part-2) # Reject obs with qiWithoutForecast < 85. OR > 100. - filter: Bounds Check @@ -244,6 +200,9 @@ - variable: name: MetaData/pressure minvalue: 50001. + action: + name: reduce obs space + - filter: Gaussian Thinning horizontal_mesh: 90 vertical_mesh: 10000 @@ -255,10 +214,62 @@ - variable: name: MetaData/pressure maxvalue: 50000. + action: + name: reduce obs space + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: false + + # Exclude data over non-water surface type where latitude > 20N for Type 253 (IRLW) --- obs tossed and not passed to setup routine + # Notes: This check was missing, so added (eliu) + # Replace land_type_index_NPOSS with water_area_fraction (eliu) + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 253 + - variable: MetaData/latitude + minvalue: 20. + test variables: + - name: GeoVaLs/water_area_fraction + minvalue: 0.99 + action: + name: reject + # Observation Post Filters (QC) + # ------------------------------ + obs post filters: # GSI setupw routine QC # Reject any ob Type [240–260] when pressure greater than 950 mb. # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + - filter: Bounds Check filter variables: - name: windEastward diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m9.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m9.yaml.j2 index 21307a34f..d5f4fac8a 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m9.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m9.yaml.j2 @@ -28,7 +28,7 @@ linear obs operator: name: VertInterp - # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI"s thinning of SEVIRI/METEOSAT-8 satwinds + # NOTE: Tests using the Gaussian Thinning filter (below) to duplicate GSI"s thinning of SEVIRI/METEOSAT-9 satwinds # results in more JEDI satwinds in the diag file than in GSI, but far fewer JEDI satwinds assimilated than # GSI. JEDI under-counts assimilated winds by roughly 25-40%, relative to GSI, and this under-count is not # even including the temporal thinning which is applied in GSI but not JEDI (by this filter below). See @@ -44,38 +44,9 @@ # round_horizontal_bin_count_to_nearest: true # partition_longitude_bins_using_mesh: true - # Observation Prior Filters (QC) - # ------------------------------ - obs prior filters: - # Apply variable changes needed for wind scaling - # For wind observations with pressure provided - - filter: Variable Transforms - Transform: SurfaceWindScalingPressure - SkipWhenNoObs: false - - # Calculate error inflation factor for duplicate observations - #- filter: Variable Assignment - # assignments: - # - name: ObsErrorFactorDuplicateCheck/windEastward - # type: float - # function: - # name: ObsFunction/ObsErrorFactorDuplicateCheck - # options: - # use_air_pressure: true - # variable: windEastward - #- filter: Variable Assignment - # assignments: - # - name: ObsErrorFactorDuplicateCheck/windNorthward - # type: float - # function: - # name: ObsFunction/ObsErrorFactorDuplicateCheck - # options: - # use_air_pressure: true - # variable: windNorthward - - # Observation Post Filters (QC) + # Observation Pre Filters (QC) # ----------------------------- - obs post filters: + obs pre filters: # Assign the initial observation error, based on height/pressure # Hard-wiring to prepobs_errtable.global by Type # ObsError is currently not updating in diag file, but passes directly to EffectiveError when no inflation is specified in YAML @@ -103,6 +74,7 @@ errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7.] + # Type 253 (MVIRI/SEVERI LWIR) - filter: Perform Action filter variables: @@ -127,6 +99,7 @@ errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4., 4.1, 5., 6., 6.3, 6.6, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7.] + # Type 254 (MVIRI/SEVIRI WV, both cloud-top and clear-sky) - filter: Perform Action filter variables: @@ -151,6 +124,7 @@ errors: [3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.8, 3.9, 3.9, 4., 4.5, 6.1, 6., 6.5, 7.3, 7.6, 7., 7.5, 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7.] + # sanity-check criteria # Observation Range Sanity Check # NOT EXPLICITLY CLEARED: No obs in this range in file, so 0 Bounds Check rejects (which is correct) but essentially untested @@ -200,24 +174,6 @@ action: name: reject - # Exclude data over non-water surface type where latitude > 20N for Type 253 (IRLW) --- obs tossed and not passed to setup routine - # Notes: This check was missing, so added (eliu) - # Replace land_type_index_NPOSS with water_area_fraction (eliu) - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 253 - - variable: MetaData/latitude - minvalue: 20. - test variables: - - name: GeoVaLs/water_area_fraction - minvalue: 0.99 - action: - name: reject - # GSI read routine QC (part-2) # Reject obs with qiWithoutForecast < 85. OR > 100. - filter: Bounds Check @@ -244,6 +200,9 @@ - variable: name: MetaData/pressure minvalue: 50001. + action: + name: reduce obs space + - filter: Gaussian Thinning horizontal_mesh: 90 vertical_mesh: 10000 @@ -255,10 +214,62 @@ - variable: name: MetaData/pressure maxvalue: 50000. + action: + name: reduce obs space + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: false + + # Exclude data over non-water surface type where latitude > 20N for Type 253 (IRLW) --- obs tossed and not passed to setup routine + # Notes: This check was missing, so added (eliu) + # Replace land_type_index_NPOSS with water_area_fraction (eliu) + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 253 + - variable: MetaData/latitude + minvalue: 20. + test variables: + - name: GeoVaLs/water_area_fraction + minvalue: 0.99 + action: + name: reject + # Observation Post Filters (QC) + # ------------------------------ + obs post filters: # GSI setupw routine QC # Reject any ob Type [240–260] when pressure greater than 950 mb. # CLEARED: minvalue/maxvalue are >=/<=, not >/<, so editing range by 1 Pa + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + - filter: Bounds Check filter variables: - name: windEastward diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_n20.yaml.j2 index 67949e6ab..32b1afe34 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_n20.yaml.j2 @@ -119,7 +119,7 @@ - variable: name: GeoVaLs/water_area_fraction maxvalue: 0.99 - reference: GeoVaLs/surface_pressure + reference: GeoVaLs/air_pressure_at_surface value: MetaData/pressure maxvalue: -20000. # within 200 hPa above surface pressure, negative p-diff action: diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_npp.yaml.j2 index 6528a59e5..2d32a2282 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_npp.yaml.j2 @@ -119,7 +119,7 @@ - variable: name: GeoVaLs/water_area_fraction maxvalue: 0.99 - reference: GeoVaLs/surface_pressure + reference: GeoVaLs/air_pressure_at_surface value: MetaData/pressure maxvalue: -20000. # within 200 hPa above surface pressure, negative p-diff action: diff --git a/parm/jcb-gdas/observations/atmosphere/scatwind_ascat_metop-a.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-a.yaml.j2 similarity index 99% rename from parm/jcb-gdas/observations/atmosphere/scatwind_ascat_metop-a.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-a.yaml.j2 index 774fef9f6..2faa053a8 100644 --- a/parm/jcb-gdas/observations/atmosphere/scatwind_ascat_metop-a.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-a.yaml.j2 @@ -46,6 +46,8 @@ use_reduced_horizontal_grid: true round_horizontal_bin_count_to_nearest: true partition_longitude_bins_using_mesh: true + action: + name: reduce obs space # Observation Prior Filters (QC) # ------------------------------ diff --git a/parm/jcb-gdas/observations/atmosphere/ascatw.ascat_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-b.yaml.j2 similarity index 99% rename from parm/jcb-gdas/observations/atmosphere/ascatw.ascat_metop-b.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-b.yaml.j2 index 88ee802ef..713b9d3b1 100644 --- a/parm/jcb-gdas/observations/atmosphere/ascatw.ascat_metop-b.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-b.yaml.j2 @@ -46,6 +46,8 @@ use_reduced_horizontal_grid: true round_horizontal_bin_count_to_nearest: true partition_longitude_bins_using_mesh: true + action: + name: reduce obs space # Observation Prior Filters (QC) # ------------------------------ diff --git a/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-c.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-c.yaml.j2 new file mode 100644 index 000000000..ec1c5e39e --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-c.yaml.j2 @@ -0,0 +1,315 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: ascatw_ascat_metop-c + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + # Observation Operator + # -------------------- + obs operator: + name: VertInterp + # Use height vertical coordinate first + # vertical coordinate: geometric_height + vertical coordinate: geopotential_height + observation vertical coordinate group: DerivedVariables + observation vertical coordinate: adjustedHeight + interpolation method: linear + hofx scaling field: SurfaceWindScalingHeight + hofx scaling field group: DerivedVariables + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: VertInterp + vertical coordinate: geopotential_height + observation vertical coordinate group: DerivedVariables + observation vertical coordinate: adjustedHeight + interpolation method: linear + + # Observation Pre Filters (QC) + # ---------------------------- + obs pre filters: + - filter: Gaussian Thinning + horizontal_mesh: 75 + use_reduced_horizontal_grid: true + round_horizontal_bin_count_to_nearest: true + partition_longitude_bins_using_mesh: true + action: + name: reduce obs space + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Apply variable changes needed for rescaled height coordinate + - filter: Variable Transforms + Transform: AdjustedHeightCoordinate + SkipWhenNoObs: false + + # Apply variable changes needed for wind scaling + - filter: Variable Transforms + Transform: SurfaceWindScalingHeight + SkipWhenNoObs: false + + # Assign the initial observation error (constant value, 1.5 m/s right now). + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error parameter: 1.5 + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + # Reject all obs with PreQC mark already set above 3 + # NOTE: All scatwinds have an automatic PreQC mark of 2 (hard-wired default from GSI) + # - filter: PreQC + # maxvalue: 3 + # action: + # name: reject + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Reject all ASCAT (Type 290) winds with tsavg <= 273.0 (surface temperature) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 290 + # - variable: GeoVaLs/surface_temperature + - variable: GeoVaLs/skin_temperature_at_surface_where_land + maxvalue: 273. + action: + name: reject + + # Reject all ASCAT (Type 290) winds with isflg /= 0 (non-water surface) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 290 + - variable: GeoVaLs/water_area_fraction + maxvalue: 0.99 + action: + name: reject + + # Reject ASCAT (Type 290) when observed component deviates from background by more than 5.0 m/s + # NOTE: This check can reject a u- or v-component of the same observation independently, which + # is fundamentally different from how GSI rejects obs (both components are rejected if + # either component fails a check). + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Arithmetic + options: + variables: + - name: ObsValue/windEastward + - name: HofX/windEastward + coefs: [1.0, -1.0] + minvalue: -5.0 + maxvalue: 5.0 + + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Arithmetic + options: + variables: + - name: ObsValue/windNorthward + - name: HofX/windNorthward + coefs: [1.0, -1.0] + minvalue: -5.0 + maxvalue: 5.0 + + # Reject OSCAT (Type 291) when observed component deviates from background by more than 6.0 m/s + # NOTE: This check can reject a u- or v-component of the same observation independently, which + # is fundamentally different from how GSI rejects obs (both components are rejected if + # either component fails a check). + - filter: Background Check + filter variables: + - name: windEastward + - name: windNorthward + threshold: 6. + absolute threshold: 6. + where: + - variable: ObsType/windEastward + is_in: 291 + action: + name: reject + + # Reject ASCAT (Type 290) when ambiguity check fails (returned value is negative) + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 290 + test variables: + - name: ObsFunction/ScatWindsAmbiguityCheck + options: + minimum_uv: 0.0001 # hard-coding a minimum-uv for transparancy, want this to basically be zero + maxvalue: 0. + action: + name: reject + + # All scatwinds must adjust errors based on ObsErrorFactorPressureCheck. + # This check will inflate errors for obs that are too close to either + # the model top or bottom. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 290-291 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 290-291 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All scatwinds subject to a gross error check. This is contained within + # the WindsSPDBCheck, although it is not exclusive to satwinds. + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [290, 291] + cgross: [5.0, 5.0] + error_min: [1.4, 1.4] + error_max: [6.1, 6.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [290, 291] + cgross: [5.0, 5.0] + error_min: [1.4, 1.4] + error_max: [6.1, 6.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don"t want to inflate errors for duplication if one + # of the duplicates should be rejected. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + # There is no across-the-board inflation for nvqc=.true. for scatwinds, presumably because for + # this inflation to take place both nvqc must be .true. AND ibeta must be >0, see: + # https://github.com/NOAA-EMC/GSI/blob/14ae595af1b03471287d322596d35c0665336e95/src/gsi/setupw.f90#L1229 + # GSI settings must have ibeta>0 for satwinds, but not for scatwinds. + # + # If the ibeta settings for scatwinds were to change while nvqc remained .true., we would extend YAML to + # an additional filter that inflates final ob-errors across-the-board by 1/0.8 = 1.25. NOTE: the nvqc setting + # is defaulted to .false. in GSI code, but is overridden in global operational configuration. See: + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this functionality were to be activated for scatwinds, you would want to include this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 290-291 + # action: + # name: inflate error + # inflation factor: 1.25 + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 290-291 + # action: + # name: inflate error + # inflation factor: 1.25 + # END OF FILTERS# + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 index 34f762711..6081fa373 100644 --- a/parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 @@ -34,6 +34,8 @@ obs bias: input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" + variables without bc: [brightnessTemperature] + channels: {{ get_satellite_variable(observation_from_jcb, "not_biascorrtd") }} variational bc: predictors: - name: constant diff --git a/parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 index cc74f9aeb..f42e07e0b 100644 --- a/parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 @@ -34,6 +34,8 @@ obs bias: input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" + variables without bc: [brightnessTemperature] + channels: {{ get_satellite_variable(observation_from_jcb, "not_biascorrtd") }} variational bc: predictors: - name: constant From 1ba3e6cadecadf6be7c56af3c2cd22c853ef0c54 Mon Sep 17 00:00:00 2001 From: Anna Shlyaeva Date: Thu, 11 Sep 2025 09:34:22 -0600 Subject: [PATCH 149/199] Make nens per MPI in parallel recentering configurable (#168) This is needed to configure resources correctly for the marine ensemble recentering. Co-authored-by: shlyaeva --- parm/jcb-gdas/algorithm/marine/soca_ens_handler.yaml.j2 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/parm/jcb-gdas/algorithm/marine/soca_ens_handler.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_ens_handler.yaml.j2 index 661e1492d..764a5b995 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_ens_handler.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_ens_handler.yaml.j2 @@ -4,7 +4,7 @@ geometry: fields metadata: ./fields_metadata.yaml nens: '{{ marine_number_ensemble_members }}' -nens per MPI task: 2 +nens per MPI task: '{{ marine_number_ensemble_members_per_mpi }}' increment variables: &vars - sea_water_potential_temperature From 616dc4de641daed45cabd56d81dd3537db60e929 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Thu, 11 Sep 2025 12:04:08 -0400 Subject: [PATCH 150/199] Add comprehensive GitHub Copilot instructions for JCB-GDAS repository (#172) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This PR adds comprehensive GitHub Copilot instructions to help coding agents work effectively in the JCB-GDAS repository. The instructions provide detailed guidance on setup, testing, validation, and common development workflows specific to this JEDI Configuration Builder client repository. ## Key Features Added **Complete Development Environment Setup** - Step-by-step instructions for cloning and configuring the main JCB repository - Validated commands for client initialization and dependency installation - Proper PYTHONPATH configuration for development work **Detailed Timing and Timeout Information** - Client initialization: 2-3 minutes with 5+ minute timeout recommendations - Integration tests: 15-30 seconds with 60+ second timeout requirements - GitHub workflows: 3-5 minutes with explicit "NEVER CANCEL" warnings - All commands tested and timed for accuracy **Comprehensive Validation Procedures** - Manual validation steps for template changes - Integration test procedures with proper setup - Configuration rendering validation through both CLI and Python API - Template syntax validation for both .yaml and .yaml.j2 files **Repository Structure Documentation** - Complete directory tree with actual file counts (64 observation templates, 17 model templates, 7 algorithm templates) - File naming conventions and template variable requirements - Component organization across algorithm/, model/, observations/, and observation_chronicle/ directories **Troubleshooting and Common Issues** - Solutions for typical development environment problems - Performance expectations and file count references - Clear guidance on when operations may appear to hang but should not be cancelled ## Implementation Details The instructions are structured to: - Prioritize imperative commands that agents can copy-paste and execute - Include explicit timeout values for all time-consuming operations - Provide fallback procedures when primary methods encounter issues - Emphasize the configuration-only nature of this repository (no build process required) All commands have been validated in the actual repository environment to ensure they work correctly for fresh repository clones. Fixes #171. --- 💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more [Copilot coding agent tips](https://gh.io/copilot-coding-agent-tips) in the docs. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: guillaumevernieres <14031856+guillaumevernieres@users.noreply.github.com> --- parm/jcb-gdas/.github/copilot-instructions.md | 171 ++++++++++++++++++ 1 file changed, 171 insertions(+) create mode 100644 parm/jcb-gdas/.github/copilot-instructions.md diff --git a/parm/jcb-gdas/.github/copilot-instructions.md b/parm/jcb-gdas/.github/copilot-instructions.md new file mode 100644 index 000000000..1d4eca1e5 --- /dev/null +++ b/parm/jcb-gdas/.github/copilot-instructions.md @@ -0,0 +1,171 @@ +# JEDI Configuration Builder - GDAS Client + +JCB-GDAS is a configuration client repository for the JEDI Configuration Builder (JCB) system used by NOAA-EMC for Global Data Assimilation System (GDAS) weather forecasting and data assimilation. It contains YAML Jinja2 templates for configuring JEDI algorithms, models, and observations. + +Always reference these instructions first and fallback to search or bash commands only when you encounter unexpected information that does not match the info here. + +## Working Effectively + +### Initial Setup and Dependencies +- Install Python 3.6+ and basic dependencies: + ```bash + python3 -m pip install --user pyyaml jinja2 click pytest + ``` + +### JCB Development Environment Setup +- Clone and set up the main JCB repository with clients: + ```bash + cd /tmp + git clone --branch develop --recursive https://github.com/NOAA-EMC/jcb.git jcb_repo # Takes 30-60 seconds + cd jcb_repo + ./jcb_client_init.py # Takes 2-3 minutes. NEVER CANCEL. Set timeout to 5+ minutes. + python3 -m pip install --user . # Takes 30-60 seconds + ``` +- The init script clones client repositories (jcb-gdas, jcb-rdas) and algorithms into `src/jcb/configuration/` +- **CRITICAL**: Always run from development directory with PYTHONPATH for local testing + +### Testing and Validation +- Run client integration tests from JCB repo with proper PYTHONPATH: + ```bash + cd /tmp/jcb_repo + PYTHONPATH=/tmp/jcb_repo/src pytest test/client_integration -v + ``` + - Takes 15-20 seconds to complete. NEVER CANCEL. + - Some tests may fail on .git directory checks - this is normal + - Set timeout to 60+ seconds for safety + +### Rendering JEDI Configurations +- Test configuration rendering using Python API: + ```bash + cd /tmp/jcb_repo + PYTHONPATH=/tmp/jcb_repo/src python3 -c " + import jcb + import yaml + config = yaml.safe_load(open('/path/to/gdas/templates.yaml')) + result = jcb.render(config) + print('Render successful') + " + ``` +- The command-line interface requires development setup: + ```bash + cd /tmp/jcb_repo + PYTHONPATH=/tmp/jcb_repo/src python3 -m jcb.driver render input.yaml output.yaml + ``` + +## Validation + +### Manual Validation Steps +- ALWAYS test configuration rendering after making changes to templates +- Run client integration tests to ensure templates are properly structured +- Validate YAML syntax for non-template files (.yaml): + ```bash + python3 -c "import yaml; yaml.safe_load(open('file.yaml'))" + ``` +- For template files (.yaml.j2), use JCB rendering to validate: + ```bash + cd /tmp/jcb_repo + PYTHONPATH=/tmp/jcb_repo/src python3 -c "import jcb, yaml; config=yaml.safe_load(open('template_dict.yaml')); jcb.render(config)" + ``` +- Check template variable consistency across related files + +### GitHub Workflow Validation +- The repository uses GitHub Actions for CI testing at `.github/workflows/run_jcb_basic_testing.yaml` +- Workflow clones main JCB repo, initializes clients, and runs integration tests +- Takes 3-5 minutes total. NEVER CANCEL. +- Set timeout to 10+ minutes for workflow commands + +## Repository Structure + +### Key Directories +- `algorithm/` - Data assimilation algorithm templates (3dvar, LETKF, etc.) + - `atmosphere/` - Atmospheric DA algorithm configurations + - `marine/`, `aero/`, `snow/` - Other domain-specific algorithms +- `model/` - Model configuration templates + - `atmosphere/` - FV3-JEDI atmospheric model configs + - Contains Jinja2 templates (.j2) for geometry, backgrounds, increments +- `observations/` - Observation operator templates + - `atmosphere/` - Satellite and conventional observation configs + - Individual files for each instrument/platform (e.g., abi_g17.yaml.j2) +- `observation_chronicle/` - Observation metadata and channel configurations +- `test/client_integration/` - Integration test templates and configurations + +### File Naming Conventions +- All template files use `.yaml.j2` extension for Jinja2 templates +- Model templates must use `_` prefix (e.g., `atmosphere_background.yaml.j2`) +- Observation files named by instrument/platform (e.g., `iasi_metop-a.yaml.j2`) + +### Template Variables +- Variables follow naming pattern: `_` +- Example: `atmosphere_background_path`, `marine_model_timestep` +- Template files in model/ directory MUST use component-prefixed variables + +## Common Tasks + +### Repository Directory Structure (for reference) +``` +/home/runner/work/jcb-gdas/jcb-gdas/ +├── .github/workflows/run_jcb_basic_testing.yaml # CI workflow +├── algorithm/ +│ ├── aero/, atmosphere/, marine/, obstats/, snow/ +│ └── atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 # Example algorithm +├── model/ +│ ├── aero/, atmosphere/, marine/, snow/ +│ └── atmosphere/atmosphere_background.yaml.j2 # Example model config +├── observations/ +│ ├── aero/, atmosphere/, atmosphere-lgetkf/, marine/, snow/ +│ └── atmosphere/iasi_metop-a.yaml.j2 # Example observation +├── observation_chronicle/ +│ ├── atmosphere/, snow/ +│ └── atmosphere/abi_g17.yaml # Example chronicle metadata +└── test/client_integration/ + ├── gdas-atmosphere-templates.yaml # Test template dictionary + └── gdas-marine-templates.yaml +``` + +### Working with Templates +- Edit `.yaml.j2` files using standard text editors +- Variables use Jinja2 syntax: `{{ variable_name }}` +- Test rendering with sample template dictionaries from `test/client_integration/` +- Always validate template syntax before committing + +### Adding New Observations +- Create new `.yaml.j2` file in appropriate `observations//` directory +- Follow existing naming patterns (instrument_platform.yaml.j2) +- Add corresponding metadata file in `observation_chronicle/` if needed +- Test rendering with existing template configurations + +### Debugging Template Issues +- Check variable naming consistency across files +- Validate YAML structure after template rendering +- Use Python API for debugging: `jcb.render(template_dict)` returns rendered config +- Review client integration test output for template validation errors + +## Important Notes + +- **NEVER CANCEL** GitHub workflow runs - they may take 5+ minutes +- **NEVER CANCEL** client initialization - takes 2-3 minutes for git clones +- **NEVER CANCEL** integration tests - they may take 15-30 seconds but can appear to hang +- This is a configuration repository - no compilation/build process required +- Changes affect JEDI experiment configurations used in weather forecasting +- Template changes require integration testing with main JCB system +- Use development JCB setup for local testing and validation + +## Troubleshooting + +### Common Issues +- **"FileNotFoundError: configuration/apps"** - Run JCB from development directory with PYTHONPATH +- **"Template key does not start with component_"** - Ensure model templates use proper prefixed variables +- **"YAML scanner error in .j2 file"** - Don't validate .j2 templates with yaml.safe_load, use JCB rendering +- **Integration tests fail on .git directory** - This is normal, test still validates templates + +### Performance Notes +- JCB client initialization: 2-3 minutes (git clones) +- Client integration tests: 15-30 seconds +- Configuration rendering: 1-5 seconds per template +- GitHub workflow: 3-5 minutes total + +### Expected File Counts +- Algorithm templates: ~10 files per component +- Model templates: ~15 files per component +- Observation templates: ~50+ files in atmosphere/ (satellites, instruments) +- Observation chronicles: ~50+ metadata files \ No newline at end of file From adadb12a75d6b86f7b92a2fffc49f69088103899 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Thu, 11 Sep 2025 15:02:37 -0400 Subject: [PATCH 151/199] Fix atmosphere outer loop templates to use prefixed ninner variables and isolate CI testing (#170) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - [x] Fixed atmosphere outer loop templates in jcb-gdas to use prefixed ninner variables - [x] Updated test configuration with corresponding template parameters - [x] Investigated CI failure cause - jcb-rdas templates need identical fixes - [x] Confirmed validation test logic is correct - requires component prefixed variables - [x] Verified jcb-gdas templates render correctly to expected JEDI YAML format - [x] **Modified GitHub workflow to isolate jcb-gdas testing from jcb-rdas** **Validation Results:** ✅ All jcb-gdas templates properly use prefixed variables (e.g., `{{atmosphere_ninner_1}}`) ✅ Templates render to correct JEDI YAML format (`ninner: 2` as applications expect) ✅ Test logic correctly validates template variable consistency across repositories ✅ **CI now tests only jcb-gdas templates, preventing cross-repository test failures** **Solution Implemented:** Modified `.github/workflows/run_jcb_basic_testing.yaml` to create a custom `jcb_clients.yaml` that only includes the current repository being tested. This prevents the CI from cloning and testing jcb-rdas templates when testing jcb-gdas changes. **Technical Details:** - Extracts client name dynamically from repository name (e.g., `jcb-gdas` → `gdas`) - Creates custom `jcb_clients.yaml` with only the current repository - Works for any jcb client repository (gdas, rdas, etc.) - Removes dependency on jcb-rdas template fixes for jcb-gdas CI to pass The jcb-gdas repository now has fully isolated testing that validates only its own templates while ensuring they render to the correct JEDI YAML format. --- 💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more [Copilot coding agent tips](https://gh.io/copilot-coding-agent-tips) in the docs. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: guillaumevernieres <14031856+guillaumevernieres@users.noreply.github.com> Co-authored-by: Guillaume Vernieres Co-authored-by: DavidNew-NOAA <134300700+DavidNew-NOAA@users.noreply.github.com> --- .../workflows/run_jcb_basic_testing.yaml | 22 ++++++++++--------- .../atmosphere_3dvar_outer_loop_1.yaml.j2 | 2 +- .../atmosphere_3dvar_outer_loop_2.yaml.j2 | 2 +- .../gdas-atmosphere-templates.yaml | 4 ++++ 4 files changed, 18 insertions(+), 12 deletions(-) diff --git a/parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml b/parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml index 102146ca5..8b0754e2f 100644 --- a/parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml +++ b/parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml @@ -91,22 +91,24 @@ jobs: git config --global core.hooksPath empty_hooks git clone --branch ${{ env.JCB_BRANCH }} --recursive $JCB_REPO jcb_repo + - name: Create custom jcb_clients.yaml for isolated testing + run: | + cd jcb_repo + # Extract the client name from the repository name (e.g., jcb-gdas -> gdas) + CLIENT_NAME=$(echo "${{ env.JCB_APP_REPO }}" | sed 's/.*jcb-//') + # Create a custom jcb_clients.yaml that only includes the current repository + cat > jcb_clients.yaml << EOF + ${CLIENT_NAME}: + git_url: ${{ env.JCB_APP_REPO }} + git_ref: ${{ env.JCB_APP_BRANCH }} + EOF + - name: Clone the clients run: | cd jcb_repo pip install pyyaml ./jcb_client_init.py - # If all the branches are found then at this point everything should be good since - # jcb_client_init will check out the correct branches. If not then the app branch needs to - # be checked out explicitly. - if [ "${{ env.JCB_BRANCH }}" != "${{ env.JCB_APP_BRANCH }}" ]; then - echo "Checking out branch ${{ env.JCB_APP_BRANCH }} for ${{ env.JCB_APP_REPO }}" - app_path=$(python jcb_client_path.py $JCB_APP_REPO) - cd $app_path - git checkout ${{ env.JCB_APP_BRANCH }} - fi - - name: Install dependencies run: | cd jcb_repo diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 index 3b2c58cb5..58a0108b8 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 @@ -1,4 +1,4 @@ -- ninner: 2 +- ninner: {{atmosphere_ninner_1}} gradient norm reduction: 1e-10 test: on geometry: diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 index 93de3f1ba..f5c449738 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 @@ -1,4 +1,4 @@ -- ninner: 4 +- ninner: {{atmosphere_ninner_2}} gradient norm reduction: 1e-10 test: on geometry: diff --git a/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml b/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml index 3ac63c49b..9f9f3ee73 100644 --- a/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml +++ b/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml @@ -81,6 +81,10 @@ atmosphere_layout_gsib_y: 2 atmosphere_forecast_length: PT6H atmosphere_forecast_timestep: PT1H +# Minimization +atmosphere_ninner_1: 2 +atmosphere_ninner_2: 4 + # Write final increment on Gaussian grid in variational atmosphere_final_increment_prefix: "./anl/atminc." From 7d95efabfc07c57a742b0d627de29a958a3f318a Mon Sep 17 00:00:00 2001 From: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com> Date: Mon, 15 Sep 2025 08:51:39 -0400 Subject: [PATCH 152/199] Add yamls for select observations to atmosphere-lgetkf (#174) --- .../atmosphere-lgetkf/gnssro_cosmic2.yaml.j2 | 104 ++++++ .../ozone.ompsnp_npp.yaml.j2 | 341 ++++++++++++++++++ .../ozone.ompstc_npp.yaml.j2 | 142 ++++++++ .../scatwnd.ascat_metop-b.yaml.j2 | 325 +++++++++++++++++ 4 files changed, 912 insertions(+) create mode 100644 parm/jcb-gdas/observations/atmosphere-lgetkf/gnssro_cosmic2.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere-lgetkf/ozone.ompsnp_npp.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere-lgetkf/ozone.ompstc_npp.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere-lgetkf/scatwnd.ascat_metop-b.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/gnssro_cosmic2.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/gnssro_cosmic2.yaml.j2 new file mode 100644 index 000000000..17d3437bf --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/gnssro_cosmic2.yaml.j2 @@ -0,0 +1,104 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: gnssrobndnbam_cosmic2 + distribution: + name: "{{distribution_type}}" + halo size: 1250e3 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsgrouping: + group variables: [ 'sequenceNumber' ] + sort variable: 'impactHeightRO' + sort order: 'ascending' + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + simulated variables: [bendingAngle] + + # Observation Operator + # -------------------- + obs operator: + name: GnssroBndNBAM + obs options: + use_compress: 1 + sr_steps: 2 + vertlayer: full + super_ref_qc: NBAM + output_diags: true + + # Observation Filters (QC) + # ------------------------ + obs filters: + # Apply gross check using pccf + # Step 0-A: Create Diagnostic Flags + # Diagnostic flag for pccf + - filter: Create Diagnostic Flags + filter variables: + - name: bendingAngle + flags: + - name: pccfCheckReject + initial value: false + force reinitialization: true + + # Step 0-B: pccf Check for CDACC data - good: 0.1-100, reject: 0 + - filter: Bounds Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/satelliteIdentifier + is_in: 265-269,750-755,44,786,820,825 + test variables: + - name: MetaData/pccf + minvalue: 0.1 + maxvalue: 100.1 + actions: + - name: set + flag: pccfCheckReject + - name: reject + + #1. gpstop + - filter: Domain Check + filter variables: + - name: bendingAngle + where: + - variable: + name: MetaData/impactHeightRO + minvalue: 0 + maxvalue: 55000.1 + action: + name: reject + #3. RONBAM cut off check + - filter: Background Check RONBAM + filter variables: + - name: bendingAngle + action: + name: reject + defer to post: true + #4. assign obs error + - filter: ROobserror + filter variables: + - name: bendingAngle + errmodel: NBAM + defer to post: true + #5. Obs error inflate + - filter: Perform Action + filter variables: + - name: bendingAngle + action: + name: RONBAMErrInflate + defer to post: true + # -------------------------------- + + # Observation Localizations (LocalEnsembleDA) + # ------------------------------------------- + obs localizations: + - localization method: Horizontal Gaspari-Cohn + lengthscale: 1250e3 + max obs: 10000 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/ozone.ompsnp_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/ozone.ompsnp_npp.yaml.j2 new file mode 100644 index 000000000..bb003b8a0 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/ozone.ompsnp_npp.yaml.j2 @@ -0,0 +1,341 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: ompsnp_npp + distribution: + name: "{{distribution_type}}" + halo size: 1250e3 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsgrouping: + group variables: ["latitude"] + sort variable: "pressure" + sort order: "ascending" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [ozoneLayer] + + #obs operator: + # name: AtmVertInterpLay + # geovals: [mole_fraction_of_ozone_in_air] + # coefficients: [0.007886131] # convert from ppmv to DU + # nlevels: [22] + + # Observation Operator + # -------------------- + obs operator: + name: ColumnRetrieval + nlayers_retrieval: 1 + tracer variables: [mole_fraction_of_ozone_in_air] + isApriori: false + isAveragingKernel: false + totalNoVertice: false + stretchVertices: none #options: top, bottom, topbottom, none + # model units coeff: 2.240013904035E-3 # this number to match the gsihofx values + model units coeff: 2.241398632746E-3 # this number to match the gsihofx values (use this) + # the actual scientific conversion factor is + # 2.1415E-3 kg[O3]/m-2 to DU + # so the name of the geovals + # is also likely wrong, as it apprears to be a mass + # fraction given the conversion factor needed + + # Observation Pre Filters (QC) + # ---------------------------- + obs pre filters: + # Observation error assignment + - filter: Perform Action + filter variables: + - name: ozoneLayer + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [0.001, 10.1325, 16.00935, 25.43258, 40.32735, 63.93607, 101.325, + 160.0935, 254.3257, 403.2735, 639.3608, 1013.25, 1600.935, 2543.258, 4032.735, + 6393.607, 10132.5, 16009.35, 25432.57, 40327.35, 63936.07, 101325] + # errors: [7.7236, 0.020, 0.020, 0.025, 0.080, 0.150, 0.056, 0.125, 0.200, + # 0.299, 0.587, 0.864, 1.547, 2.718, 3.893, 4.353, 3.971, 4.407, 4.428, + # 3.312, 2.198, 2.285] + errors: [7.7236, 0.020, 0.020, 0.025, 0.040, 0.080, 0.156, 0.245, 0.510, + 1.098, 3.917, 6.124, 6.347, 5.798, 6.843, 9.253,10.091,10.967, 8.478, + 5.572, 2.638, 3.525] # operational from gfs.v16.3.9 (late 2023) + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Do not assimilation where pressure is zero + # Zero pressure indicates the data is total column ozone + - filter: RejectList + filter variables: + - name: ozoneLayer + where: + - variable: + name: MetaData/pressure + maxvalue: 0.0001 + + # Sanity check on observaton values + - filter: Bounds Check + filter variables: + - name: ozoneLayer + minvalue: 0 + maxvalue: 1000 + action: + name: reject + + # Total Ozone Quality Check (keeps 0, 2) + # 0 indentifies good data + # 2 identifies good data with a solar zenith angle > 84 degrees + - filter: RejectList + filter variables: + - name: ozoneLayer + where: + - variable: + name: MetaData/totalOzoneQuality + is_not_in: 0, 2 + + # Profile Ozone Quality Check (keeps 0, 1, 7) + # 0 : good data + # 1 : good data with a solar zenith angle > 84 degrees + # 7 : profile for which stray light correction applied + - filter: RejectList + filter variables: + - name: ozoneLayer + where: + - variable: + name: MetaData/profileOzoneQuality + is_not_in: 0, 1, 7 + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Gross error check + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 120 + action: + name: reject + where: + - variable: + name: MetaData/pressure + maxvalue: 0.001 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 30 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 30000.0 + maxvalue: 110000.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 40 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 20000.0 + maxvalue: 30000.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 44.42 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 10100.0 + maxvalue: 20000.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 57.52 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 6400.0 + maxvalue: 10100.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 69.4 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 4000.0 + maxvalue: 6400.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 70 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 2600.0 + maxvalue: 4000.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 62.73 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 1600.0 + maxvalue: 2600.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 50.52 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 1100.0 + maxvalue: 1600.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 35.9 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 700.0 + maxvalue: 1100.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 26.41 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 400.0 + maxvalue: 700.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 20.51 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 300.0 + maxvalue: 400.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 12.82 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 200.0 + maxvalue: 300.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 10 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 70.0 + maxvalue: 200.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 5 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 40.0 + maxvalue: 70.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 2 + action: + name: reject + where: + - variable: + name: MetaData/pressure + minvalue: 30.0 + maxvalue: 40.0 + + - filter: Background Check + filter variables: + - name: ozoneLayer + absolute threshold: 1 + action: + name: reject + where: + - variable: + name: MetaData/pressure + maxvalue: 30.0 + + # End of Filters + + # Observation Localizations (LocalEnsembleDA) + # ------------------------------------------- + obs localizations: + - localization method: Horizontal Gaspari-Cohn + lengthscale: 1250e3 + max nobs: 10000 + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/ozone.ompstc_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/ozone.ompstc_npp.yaml.j2 new file mode 100644 index 000000000..78904b9fc --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/ozone.ompstc_npp.yaml.j2 @@ -0,0 +1,142 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: ompstc_npp + distribution: + name: "{{distribution_type}}" + halo size: 1250e3 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [ozoneTotal] + + #obs operator: + # name: AtmVertInterpLay + # geovals: [mole_fraction_of_ozone_in_air] + # coefficients: [0.007886131] # convert from ppmv to DU + # nlevels: [1] + + # Observation Operator + # -------------------- + obs operator: + name: ColumnRetrieval + nlayers_retrieval: 1 + tracer variables: [mole_fraction_of_ozone_in_air] + isApriori: false + isAveragingKernel: false + totalNoVertice: true + stretchVertices: topbottom # options: top, bottom, topbottom, none + model units coeff: 2.241398632746E-3 + + # Observation Pre Filters (QC) + # ---------------------------- + obs pre filters: + # assign observation error + - filter: Perform Action + filter variables: + - name: ozoneTotal + action: + name: assign error + error parameter: 6.0 + + # GSI read routine QC + # range sanity check + - filter: Bounds Check + filter variables: + - name: ozoneTotal + minvalue: 0 + maxvalue: 1000 + action: + name: reject + + #- filter: Gaussian Thinning + # horizontal_mesh: 150 + # use_reduced_horizontal_grid: true + # distance_norm: geodesic + # action: + # name: reject + + # Accept total_ozone_error_flag values of 0 and 1, but not any others. + - filter: RejectList + filter variables: + - name: ozoneTotal + where: + - variable: + name: MetaData/totalOzoneQualityCode + is_not_in: 0, 1 + + - filter: RejectList + filter variables: + - name: ozoneTotal + where: + - variable: + name: MetaData/bestOzoneAlgorithmFlag + is_in: 3, 13 + + # GSI setup routine QC + - filter: RejectList + filter variables: + - name: ozoneTotal + where: + - variable: + name: MetaData/sensorScanPosition + is_in: 1, 2, 3, 4, 35 + - variable: + name: MetaData/latitude + minvalue: 50.0 + + - filter: RejectList + filter variables: + - name: ozoneTotal + where: + - variable: + name: MetaData/sensorScanPosition + is_in: 1, 2, 3, 4, 35 + - variable: + name: MetaData/latitude + maxvalue: -50.0 + + - filter: Gaussian Thinning + horizontal_mesh: 150 + use_reduced_horizontal_grid: true + distance_norm: geodesic + action: + name: reduce obs space + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + - filter: Background Check + filter variables: + - name: ozoneTotal + threshold: 10.0 + absolute threshold: 300.0 + action: + name: reject + + # End of Filters + + # Observation Localizations (LocalEnsembleDA) + # ------------------------------------------- + obs localizations: + - localization method: Horizontal Gaspari-Cohn + lengthscale: 1250e3 + max nobs: 10000 + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/scatwnd.ascat_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/scatwnd.ascat_metop-b.yaml.j2 new file mode 100644 index 000000000..5a63e4310 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/scatwnd.ascat_metop-b.yaml.j2 @@ -0,0 +1,325 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: ascatw_ascat_metop-b + distribution: + name: "{{distribution_type}}" + halo size: 1250e3 + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + # Observation Operator + # -------------------- + obs operator: + name: VertInterp + # Use height vertical coordinate first + # vertical coordinate: geometric_height + vertical coordinate: geopotential_height + observation vertical coordinate group: DerivedVariables + observation vertical coordinate: adjustedHeight + interpolation method: linear + hofx scaling field: SurfaceWindScalingHeight + hofx scaling field group: DerivedVariables + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: VertInterp + vertical coordinate: geopotential_height + observation vertical coordinate group: DerivedVariables + observation vertical coordinate: adjustedHeight + interpolation method: linear + + # Observation Pre Filters (QC) + # ---------------------------- + obs pre filters: + - filter: Gaussian Thinning + horizontal_mesh: 75 + use_reduced_horizontal_grid: true + round_horizontal_bin_count_to_nearest: true + partition_longitude_bins_using_mesh: true + action: + name: reduce obs space + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Apply variable changes needed for rescaled height coordinate + - filter: Variable Transforms + Transform: AdjustedHeightCoordinate + SkipWhenNoObs: false + + # Apply variable changes needed for wind scaling + - filter: Variable Transforms + Transform: SurfaceWindScalingHeight + SkipWhenNoObs: false + + # Assign the initial observation error (constant value, 1.5 m/s right now). + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error parameter: 1.5 + + # Calculate error inflation factor for duplicate observations + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windEastward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windEastward + #- filter: Variable Assignment + # assignments: + # - name: ObsErrorFactorDuplicateCheck/windNorthward + # type: float + # function: + # name: ObsFunction/ObsErrorFactorDuplicateCheck + # options: + # use_air_pressure: true + # variable: windNorthward + # Reject all obs with PreQC mark already set above 3 + # NOTE: All scatwinds have an automatic PreQC mark of 2 (hard-wired default from GSI) + # - filter: PreQC + # maxvalue: 3 + # action: + # name: reject + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Reject all ASCAT (Type 290) winds with tsavg <= 273.0 (surface temperature) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 290 + # - variable: GeoVaLs/surface_temperature + - variable: GeoVaLs/skin_temperature_at_surface_where_land + maxvalue: 273. + action: + name: reject + + # Reject all ASCAT (Type 290) winds with isflg /= 0 (non-water surface) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 290 + - variable: GeoVaLs/water_area_fraction + maxvalue: 0.99 + action: + name: reject + + # Reject ASCAT (Type 290) when observed component deviates from background by more than 5.0 m/s + # NOTE: This check can reject a u- or v-component of the same observation independently, which + # is fundamentally different from how GSI rejects obs (both components are rejected if + # either component fails a check). + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Arithmetic + options: + variables: + - name: ObsValue/windEastward + - name: HofX/windEastward + coefs: [1.0, -1.0] + minvalue: -5.0 + maxvalue: 5.0 + + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + test variables: + - name: ObsFunction/Arithmetic + options: + variables: + - name: ObsValue/windNorthward + - name: HofX/windNorthward + coefs: [1.0, -1.0] + minvalue: -5.0 + maxvalue: 5.0 + + # Reject OSCAT (Type 291) when observed component deviates from background by more than 6.0 m/s + # NOTE: This check can reject a u- or v-component of the same observation independently, which + # is fundamentally different from how GSI rejects obs (both components are rejected if + # either component fails a check). + - filter: Background Check + filter variables: + - name: windEastward + - name: windNorthward + threshold: 6. + absolute threshold: 6. + where: + - variable: ObsType/windEastward + is_in: 291 + action: + name: reject + + # Reject ASCAT (Type 290) when ambiguity check fails (returned value is negative) + - filter: Bounds Check + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: ObsType/windEastward + is_in: 290 + test variables: + - name: ObsFunction/ScatWindsAmbiguityCheck + options: + minimum_uv: 0.0001 # hard-coding a minimum-uv for transparancy, want this to basically be zero + maxvalue: 0. + action: + name: reject + + # All scatwinds must adjust errors based on ObsErrorFactorPressureCheck. + # This check will inflate errors for obs that are too close to either + # the model top or bottom. + - filter: Perform Action + filter variables: + - name: windEastward + where: + - variable: + name: ObsType/windEastward + is_in: 290-291 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windEastward + inflation factor: 4.0 + + - filter: Perform Action + filter variables: + - name: windNorthward + where: + - variable: + name: ObsType/windNorthward + is_in: 290-291 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + + # All scatwinds subject to a gross error check. This is contained within + # the WindsSPDBCheck, although it is not exclusive to satwinds. + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [290, 291] + cgross: [5.0, 5.0] + error_min: [1.4, 1.4] + error_max: [6.1, 6.1] + variable: windEastward + action: + name: reject + + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [290, 291] + cgross: [5.0, 5.0] + error_min: [1.4, 1.4] + error_max: [6.1, 6.1] + variable: windNorthward + action: + name: reject + + # The last error inflation check is for duplicate observations. This one needs + # to come last, because we don"t want to inflate errors for duplication if one + # of the duplicates should be rejected. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windEastward + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # action: + # name: inflate error + # inflation variable: + # name: ObsErrorFactorDuplicateCheck/windNorthward + # There is no across-the-board inflation for nvqc=.true. for scatwinds, presumably because for + # this inflation to take place both nvqc must be .true. AND ibeta must be >0, see: + # https://github.com/NOAA-EMC/GSI/blob/14ae595af1b03471287d322596d35c0665336e95/src/gsi/setupw.f90#L1229 + # GSI settings must have ibeta>0 for satwinds, but not for scatwinds. + # + # If the ibeta settings for scatwinds were to change while nvqc remained .true., we would extend YAML to + # an additional filter that inflates final ob-errors across-the-board by 1/0.8 = 1.25. NOTE: the nvqc setting + # is defaulted to .false. in GSI code, but is overridden in global operational configuration. See: + # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 + # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in + # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. + # + # If this functionality were to be activated for scatwinds, you would want to include this last inflation filter. + #- filter: Perform Action + # filter variables: + # - name: windEastward + # where: + # - variable: ObsType/windEastward + # is_in: 290-291 + # action: + # name: inflate error + # inflation factor: 1.25 + #- filter: Perform Action + # filter variables: + # - name: windNorthward + # where: + # - variable: ObsType/windNorthward + # is_in: 290-291 + # action: + # name: inflate error + # inflation factor: 1.25 + # END OF FILTERS# + + + # Observation Localizations (LocalEnsembleDA) + # ------------------------------------------- + obs localizations: + - localization method: Horizontal Gaspari-Cohn + lengthscale: 1250e3 + max nobs: 10000 + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 From 8cbc5da830d715c6b5efd6fdf8f53807c9fc1c4f Mon Sep 17 00:00:00 2001 From: DavidNew-NOAA <134300700+DavidNew-NOAA@users.noreply.github.com> Date: Mon, 15 Sep 2025 10:40:51 -0400 Subject: [PATCH 153/199] Revert unnecessary copilot changes (#176) This PR reverts some of that changes made in https://github.com/NOAA-EMC/jcb-gdas/pull/170 . These were changes were from copilot's initial changes meant to fix CI testing via GitHub Actions. A later round of changes fixed the CI, but the other changes remain. Without this PR, certain atmospheric JEDI yamls will fail to render with JCB due to the absence of parameters in jcb-base.yaml.j2 --- .../model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 | 2 +- .../model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 | 2 +- .../test/client_integration/gdas-atmosphere-templates.yaml | 7 ------- 3 files changed, 2 insertions(+), 9 deletions(-) diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 index 58a0108b8..3b2c58cb5 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 @@ -1,4 +1,4 @@ -- ninner: {{atmosphere_ninner_1}} +- ninner: 2 gradient norm reduction: 1e-10 test: on geometry: diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 index f5c449738..93de3f1ba 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 @@ -1,4 +1,4 @@ -- ninner: {{atmosphere_ninner_2}} +- ninner: 4 gradient norm reduction: 1e-10 test: on geometry: diff --git a/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml b/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml index 9f9f3ee73..f6a8dd11c 100644 --- a/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml +++ b/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml @@ -69,26 +69,19 @@ atmosphere_ensemble_analysis_prefix: "anl_" atmosphere_background_time_iso: '2024-02-02T00:00:00Z' # Background error - atmosphere_bump_data_directory: DATA/berror atmosphere_gsibec_path: DATA/berror atmosphere_number_ensemble_members: 3 atmosphere_layout_gsib_x: 2 atmosphere_layout_gsib_y: 2 - # Forecasting atmosphere_forecast_length: PT6H atmosphere_forecast_timestep: PT1H -# Minimization -atmosphere_ninner_1: 2 -atmosphere_ninner_2: 4 - # Write final increment on Gaussian grid in variational atmosphere_final_increment_prefix: "./anl/atminc." - # Observation things # ------------------ observations: all_observations From fdbfdd3d71bf755346a7aa589181a034244c7209 Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Mon, 15 Sep 2025 10:48:21 -0400 Subject: [PATCH 154/199] make changes to Psfc template for ioda-stats (#175) Update the conventional Psfc ioda-stats Jinja2 template to look like the snow templates, so that it should actually run properly given code changes since this file was last updated. --- .../obstats/atmos/conventional_ps_template.yaml.j2 | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/parm/jcb-gdas/algorithm/obstats/atmos/conventional_ps_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/atmos/conventional_ps_template.yaml.j2 index b187ebf62..57e77fc28 100644 --- a/parm/jcb-gdas/algorithm/obstats/atmos/conventional_ps_template.yaml.j2 +++ b/parm/jcb-gdas/algorithm/obstats/atmos/conventional_ps_template.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: sfc + name: prepbufr_adpsfc obsdatain: engine: type: H5File @@ -7,7 +7,13 @@ simulated variables: ['stationPressure'] observed variables: ['stationPressure'] statistics to compute: ['mean', 'count', 'RMS'] + variables: ['stationPressure'] + groups to process: ['ombg', 'oman'] + qc groups: ['EffectiveQC0', 'EffectiveQC1'] output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_atmos.nc" + regular grid binning: + bin size in degrees: 1.0 + use negative longitudes: true domains to process: - domain: name: "SH" From 21a102f8f855a8230a199ac1240f1e36c13ea4e4 Mon Sep 17 00:00:00 2001 From: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com> Date: Thu, 18 Sep 2025 06:59:59 -0400 Subject: [PATCH 155/199] update surface pressure correction scheme keywords (#178) --- .../observations/atmosphere-lgetkf/conventional_ps.yaml.j2 | 4 ++-- parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 | 4 ++-- parm/jcb-gdas/observations/atmosphere/sfc.yaml.j2 | 4 ++-- parm/jcb-gdas/observations/atmosphere/sfcship.yaml.j2 | 4 ++-- parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 | 4 ++-- 5 files changed, 10 insertions(+), 10 deletions(-) diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml.j2 index 49f99a5a9..54b3241d3 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml.j2 @@ -22,10 +22,10 @@ # Observation Operator # -------------------- obs operator: - name: SfcPCorrected + name: SfcCorrected variables: - name: stationPressure - da_psfc_scheme: GSI + correction scheme to use: GSL station_altitude: height geovar_sfc_geomz: height_above_mean_sea_level_at_surface geovar_geomz: geopotential_height diff --git a/parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 index 225a485dd..ad79038c0 100644 --- a/parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 @@ -19,10 +19,10 @@ # Observation Operator # -------------------- obs operator: - name: SfcPCorrected + name: SfcCorrected variables: - name: stationPressure - da_psfc_scheme: GSI + correction scheme to use: GSL station_altitude: height geovar_sfc_geomz: height_above_mean_sea_level_at_surface geovar_geomz: geopotential_height diff --git a/parm/jcb-gdas/observations/atmosphere/sfc.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/sfc.yaml.j2 index b470a0eac..f659d7dda 100644 --- a/parm/jcb-gdas/observations/atmosphere/sfc.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/sfc.yaml.j2 @@ -19,8 +19,8 @@ # Observation Operator # -------------------- obs operator: - name: SfcPCorrected - da_psfc_scheme: GSI + name: SfcCorrected + correction scheme to use: GSL geovar_sfc_geomz: surface_geometric_height geovar_geomz: geopotential_height diff --git a/parm/jcb-gdas/observations/atmosphere/sfcship.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/sfcship.yaml.j2 index 5e1a227d4..6d0c4980a 100644 --- a/parm/jcb-gdas/observations/atmosphere/sfcship.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/sfcship.yaml.j2 @@ -26,10 +26,10 @@ variables: - name: airTemperature - name: specificHumidity - - name: SfcPCorrected + - name: SfcCorrected variables: - name: stationPressure - da_psfc_scheme: GSI + correction scheme to use: GSL geovar_sfc_geomz: surface_geometric_height geovar_geomz: geopotential_height diff --git a/parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 index 224c81459..e965d865d 100644 --- a/parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 @@ -32,8 +32,8 @@ - name: windEastward - name: windNorthward - name: specificHumidity - - name: SfcPCorrected - da_psfc_scheme: GSI + - name: SfcCorrected + correction scheme to use: GSL geovar_sfc_geomz: geopotential_height_at_surface geovar_geomz: geopotential_height variables: From 5606433e10d7bbbb442a8112db149439255d7323 Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Fri, 19 Sep 2025 14:18:12 -0400 Subject: [PATCH 156/199] Make the MPI layout for the IMS preprocessor variable (#179) Make the layout for the IMS preprocessor variable --- parm/jcb-gdas/algorithm/snow/snow_ims_scf_preprocess.yaml.j2 | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/parm/jcb-gdas/algorithm/snow/snow_ims_scf_preprocess.yaml.j2 b/parm/jcb-gdas/algorithm/snow/snow_ims_scf_preprocess.yaml.j2 index 987234355..67ac7276a 100644 --- a/parm/jcb-gdas/algorithm/snow/snow_ims_scf_preprocess.yaml.j2 +++ b/parm/jcb-gdas/algorithm/snow/snow_ims_scf_preprocess.yaml.j2 @@ -3,7 +3,9 @@ geometry: namelist filename: ./fv3jedi/fmsmpp.nml field table filename: ./fv3jedi/field_table akbk: ./fv3jedi/akbk.nc4 - layout: [1, 1] + layout: + - {{snow_layout_x}} + - {{snow_layout_y}} npx: {{ snow_npx_ges }} npy: {{ snow_npy_ges }} npz: {{ snow_npz_ges }} From 6337550eb7fbd63e68be7d8572a5f3c75b19fa18 Mon Sep 17 00:00:00 2001 From: Andy Tangborn <71304161+andytangborn@users.noreply.github.com> Date: Tue, 23 Sep 2025 15:02:06 -0400 Subject: [PATCH 157/199] Update jcb-gdas observation yamls (#180) This PR adds a warning (rather than failing) for missing obs files and increases the maximum pool size to 2. Co-authored-by: Cory Martin --- parm/jcb-gdas/observations/aero/viirs_n20_aod.yaml.j2 | 3 ++- parm/jcb-gdas/observations/aero/viirs_n21_aod.yaml.j2 | 3 ++- parm/jcb-gdas/observations/aero/viirs_npp_aod.yaml.j2 | 3 ++- 3 files changed, 6 insertions(+), 3 deletions(-) diff --git a/parm/jcb-gdas/observations/aero/viirs_n20_aod.yaml.j2 b/parm/jcb-gdas/observations/aero/viirs_n20_aod.yaml.j2 index 5ef6aca35..06613f4a4 100644 --- a/parm/jcb-gdas/observations/aero/viirs_n20_aod.yaml.j2 +++ b/parm/jcb-gdas/observations/aero/viirs_n20_aod.yaml.j2 @@ -8,12 +8,13 @@ engine: type: H5File obsfile: "{{aero_obsdatain_path}}/{{aero_obsdatain_prefix}}{{observation_from_jcb}}{{aero_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File obsfile: "{{aero_obsdataout_path}}/{{aero_obsdataout_prefix}}{{observation_from_jcb}}{{aero_obsdataout_suffix}}" io pool: - max pool size: 1 + max pool size: 2 simulated variables: [aerosolOpticalDepth] channels: 4 diff --git a/parm/jcb-gdas/observations/aero/viirs_n21_aod.yaml.j2 b/parm/jcb-gdas/observations/aero/viirs_n21_aod.yaml.j2 index 9ea41402b..87119ad52 100644 --- a/parm/jcb-gdas/observations/aero/viirs_n21_aod.yaml.j2 +++ b/parm/jcb-gdas/observations/aero/viirs_n21_aod.yaml.j2 @@ -8,12 +8,13 @@ engine: type: H5File obsfile: "{{aero_obsdatain_path}}/{{aero_obsdatain_prefix}}{{observation_from_jcb}}{{aero_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File obsfile: "{{aero_obsdataout_path}}/{{aero_obsdataout_prefix}}{{observation_from_jcb}}{{aero_obsdataout_suffix}}" io pool: - max pool size: 1 + max pool size: 2 simulated variables: [aerosolOpticalDepth] channels: 4 diff --git a/parm/jcb-gdas/observations/aero/viirs_npp_aod.yaml.j2 b/parm/jcb-gdas/observations/aero/viirs_npp_aod.yaml.j2 index 57cad00a1..fc28d42e6 100644 --- a/parm/jcb-gdas/observations/aero/viirs_npp_aod.yaml.j2 +++ b/parm/jcb-gdas/observations/aero/viirs_npp_aod.yaml.j2 @@ -12,8 +12,9 @@ engine: type: H5File obsfile: "{{aero_obsdataout_path}}/{{aero_obsdataout_prefix}}{{observation_from_jcb}}{{aero_obsdataout_suffix}}" + missing file action: warn io pool: - max pool size: 1 + max pool size: 2 simulated variables: [aerosolOpticalDepth] channels: 4 From cf164d4445df0ac1e842ee3670225b06a164a64a Mon Sep 17 00:00:00 2001 From: Yaping Wang <49168260+ypwang19@users.noreply.github.com> Date: Wed, 24 Sep 2025 13:27:51 -0400 Subject: [PATCH 158/199] Skip copy if aero obs file is missing (#181) This PR updates the yaml to skip copying when a source aerosol observation file is missing. Co-authored-by: ypwang19 --- parm/jcb-gdas/algorithm/aero/aero_obs_staging.yaml.j2 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/parm/jcb-gdas/algorithm/aero/aero_obs_staging.yaml.j2 b/parm/jcb-gdas/algorithm/aero/aero_obs_staging.yaml.j2 index 0e5c9920b..7b704cb21 100644 --- a/parm/jcb-gdas/algorithm/aero/aero_obs_staging.yaml.j2 +++ b/parm/jcb-gdas/algorithm/aero/aero_obs_staging.yaml.j2 @@ -1,6 +1,6 @@ mkdir: - '{{aero_obsdatain_path}}' -copy: +copy_opt: {% for observation_from_jcb in observations %} {% if use_observer(observation_from_jcb) %} - ['{{aero_obsdataroot_path}}/{{aero_obsdatain_prefix}}{{observation_from_jcb}}{{aero_obsdatain_suffix}}', '{{aero_obsdatain_path}}'] From 191b574394c67f035503f0586174ce39723ba691 Mon Sep 17 00:00:00 2001 From: Guillaume Vernieres Date: Mon, 29 Sep 2025 13:12:48 -0400 Subject: [PATCH 159/199] Clean-up of the ice concentration obs yamls (#182) I realized that we were missing a few obs configs while testing obsForge, so here they are as links to a generic yaml for passive ice concentration. --- .../marine/icec_abi_g16_l2.yaml.j2 | 68 +------------------ .../marine/icec_amsu_ma1_l2.yaml.j2 | 1 + .../marine/icec_amsu_mb_l2.yaml.j2 | 68 +------------------ .../marine/icec_atms_n20_l2.yaml.j2 | 68 +------------------ .../marine/icec_atms_n21_l2.yaml.j2 | 68 +------------------ .../marine/icec_atms_npp_l2.yaml.j2 | 68 +------------------ .../marine/icec_generic_passive.yaml.j2 | 67 ++++++++++++++++++ .../marine/icec_gmi_gpm_l2.yaml.j2 | 62 +---------------- .../marine/icec_viirs_j01_l2.yaml.j2 | 1 + .../marine/icec_viirs_n20_l2_north.yaml.j2 | 68 +------------------ .../marine/icec_viirs_n20_l2_south.yaml.j2 | 68 +------------------ .../marine/icec_viirs_n21_l2.yaml.j2 | 1 + .../marine/icec_viirs_npp_l2.yaml.j2 | 1 + 13 files changed, 79 insertions(+), 530 deletions(-) mode change 100644 => 120000 parm/jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 create mode 120000 parm/jcb-gdas/observations/marine/icec_amsu_ma1_l2.yaml.j2 mode change 100644 => 120000 parm/jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 mode change 100644 => 120000 parm/jcb-gdas/observations/marine/icec_atms_n20_l2.yaml.j2 mode change 100644 => 120000 parm/jcb-gdas/observations/marine/icec_atms_n21_l2.yaml.j2 mode change 100644 => 120000 parm/jcb-gdas/observations/marine/icec_atms_npp_l2.yaml.j2 create mode 100644 parm/jcb-gdas/observations/marine/icec_generic_passive.yaml.j2 mode change 100644 => 120000 parm/jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 create mode 120000 parm/jcb-gdas/observations/marine/icec_viirs_j01_l2.yaml.j2 mode change 100644 => 120000 parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_north.yaml.j2 mode change 100644 => 120000 parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_south.yaml.j2 create mode 120000 parm/jcb-gdas/observations/marine/icec_viirs_n21_l2.yaml.j2 create mode 120000 parm/jcb-gdas/observations/marine/icec_viirs_npp_l2.yaml.j2 diff --git a/parm/jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 deleted file mode 100644 index 7c6cbdc66..000000000 --- a/parm/jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 +++ /dev/null @@ -1,67 +0,0 @@ -- obs space: - name: {{observation_from_jcb}} - obsdatain: - engine: - type: H5File - obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" - obsdataout: - engine: - type: H5File - obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" - simulated variables: [seaIceFraction] - io pool: - max pool size: 1 - get values: - time interpolation: linear - obs operator: - name: Identity - observation alias file: obsop_name_map.yaml - obs error: - covariance model: diagonal - obs filters: - - filter: Domain Check - where: - - variable: {name: GeoVaLs/sea_area_fraction} - value: is_valid - minvalue: 0.9 - - filter: Bounds Check - minvalue: 0.0 - maxvalue: 1.0 - #- filter: Background Check - # threshold: 5.0 - - filter: Domain Check - where: - - variable: { name: GeoVaLs/sea_surface_temperature} - maxvalue: 2.0 - - filter: Domain Check - where: - - variable: { name: GeoVaLs/sea_surface_temperature} - maxvalue: 0.0 - action: - name: inflate error - inflation factor: 2.0 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 0e3 - - filter: Gaussian Thinning - horizontal_mesh: 25 #km - use_reduced_horizontal_grid: true - select_median: true - action: - name: reduce obs space -# - filter: Domain Check -# action: -# name: passivate -# where: -# - variable: {name: GeoVaLs/sea_surface_temperature} -# maxvalue: -4.0 -{% if letkf_app | default(false) %} - obs localizations: - - localization method: Rossby - base value: 100.0e3 - rossby mult: 1.0 - min grid mult: 2.0 - min value: 200.0e3 - max value: 900.0e3 -{% endif %} diff --git a/parm/jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 new file mode 120000 index 000000000..3ebe0973c --- /dev/null +++ b/parm/jcb-gdas/observations/marine/icec_abi_g16_l2.yaml.j2 @@ -0,0 +1 @@ +icec_generic_passive.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/icec_amsu_ma1_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_amsu_ma1_l2.yaml.j2 new file mode 120000 index 000000000..3ebe0973c --- /dev/null +++ b/parm/jcb-gdas/observations/marine/icec_amsu_ma1_l2.yaml.j2 @@ -0,0 +1 @@ +icec_generic_passive.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 deleted file mode 100644 index 7c6cbdc66..000000000 --- a/parm/jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 +++ /dev/null @@ -1,67 +0,0 @@ -- obs space: - name: {{observation_from_jcb}} - obsdatain: - engine: - type: H5File - obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" - obsdataout: - engine: - type: H5File - obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" - simulated variables: [seaIceFraction] - io pool: - max pool size: 1 - get values: - time interpolation: linear - obs operator: - name: Identity - observation alias file: obsop_name_map.yaml - obs error: - covariance model: diagonal - obs filters: - - filter: Domain Check - where: - - variable: {name: GeoVaLs/sea_area_fraction} - value: is_valid - minvalue: 0.9 - - filter: Bounds Check - minvalue: 0.0 - maxvalue: 1.0 - #- filter: Background Check - # threshold: 5.0 - - filter: Domain Check - where: - - variable: { name: GeoVaLs/sea_surface_temperature} - maxvalue: 2.0 - - filter: Domain Check - where: - - variable: { name: GeoVaLs/sea_surface_temperature} - maxvalue: 0.0 - action: - name: inflate error - inflation factor: 2.0 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 0e3 - - filter: Gaussian Thinning - horizontal_mesh: 25 #km - use_reduced_horizontal_grid: true - select_median: true - action: - name: reduce obs space -# - filter: Domain Check -# action: -# name: passivate -# where: -# - variable: {name: GeoVaLs/sea_surface_temperature} -# maxvalue: -4.0 -{% if letkf_app | default(false) %} - obs localizations: - - localization method: Rossby - base value: 100.0e3 - rossby mult: 1.0 - min grid mult: 2.0 - min value: 200.0e3 - max value: 900.0e3 -{% endif %} diff --git a/parm/jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 new file mode 120000 index 000000000..3ebe0973c --- /dev/null +++ b/parm/jcb-gdas/observations/marine/icec_amsu_mb_l2.yaml.j2 @@ -0,0 +1 @@ +icec_generic_passive.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/icec_atms_n20_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_atms_n20_l2.yaml.j2 deleted file mode 100644 index 7c6cbdc66..000000000 --- a/parm/jcb-gdas/observations/marine/icec_atms_n20_l2.yaml.j2 +++ /dev/null @@ -1,67 +0,0 @@ -- obs space: - name: {{observation_from_jcb}} - obsdatain: - engine: - type: H5File - obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" - obsdataout: - engine: - type: H5File - obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" - simulated variables: [seaIceFraction] - io pool: - max pool size: 1 - get values: - time interpolation: linear - obs operator: - name: Identity - observation alias file: obsop_name_map.yaml - obs error: - covariance model: diagonal - obs filters: - - filter: Domain Check - where: - - variable: {name: GeoVaLs/sea_area_fraction} - value: is_valid - minvalue: 0.9 - - filter: Bounds Check - minvalue: 0.0 - maxvalue: 1.0 - #- filter: Background Check - # threshold: 5.0 - - filter: Domain Check - where: - - variable: { name: GeoVaLs/sea_surface_temperature} - maxvalue: 2.0 - - filter: Domain Check - where: - - variable: { name: GeoVaLs/sea_surface_temperature} - maxvalue: 0.0 - action: - name: inflate error - inflation factor: 2.0 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 0e3 - - filter: Gaussian Thinning - horizontal_mesh: 25 #km - use_reduced_horizontal_grid: true - select_median: true - action: - name: reduce obs space -# - filter: Domain Check -# action: -# name: passivate -# where: -# - variable: {name: GeoVaLs/sea_surface_temperature} -# maxvalue: -4.0 -{% if letkf_app | default(false) %} - obs localizations: - - localization method: Rossby - base value: 100.0e3 - rossby mult: 1.0 - min grid mult: 2.0 - min value: 200.0e3 - max value: 900.0e3 -{% endif %} diff --git a/parm/jcb-gdas/observations/marine/icec_atms_n20_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_atms_n20_l2.yaml.j2 new file mode 120000 index 000000000..3ebe0973c --- /dev/null +++ b/parm/jcb-gdas/observations/marine/icec_atms_n20_l2.yaml.j2 @@ -0,0 +1 @@ +icec_generic_passive.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/icec_atms_n21_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_atms_n21_l2.yaml.j2 deleted file mode 100644 index 7c6cbdc66..000000000 --- a/parm/jcb-gdas/observations/marine/icec_atms_n21_l2.yaml.j2 +++ /dev/null @@ -1,67 +0,0 @@ -- obs space: - name: {{observation_from_jcb}} - obsdatain: - engine: - type: H5File - obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" - obsdataout: - engine: - type: H5File - obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" - simulated variables: [seaIceFraction] - io pool: - max pool size: 1 - get values: - time interpolation: linear - obs operator: - name: Identity - observation alias file: obsop_name_map.yaml - obs error: - covariance model: diagonal - obs filters: - - filter: Domain Check - where: - - variable: {name: GeoVaLs/sea_area_fraction} - value: is_valid - minvalue: 0.9 - - filter: Bounds Check - minvalue: 0.0 - maxvalue: 1.0 - #- filter: Background Check - # threshold: 5.0 - - filter: Domain Check - where: - - variable: { name: GeoVaLs/sea_surface_temperature} - maxvalue: 2.0 - - filter: Domain Check - where: - - variable: { name: GeoVaLs/sea_surface_temperature} - maxvalue: 0.0 - action: - name: inflate error - inflation factor: 2.0 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 0e3 - - filter: Gaussian Thinning - horizontal_mesh: 25 #km - use_reduced_horizontal_grid: true - select_median: true - action: - name: reduce obs space -# - filter: Domain Check -# action: -# name: passivate -# where: -# - variable: {name: GeoVaLs/sea_surface_temperature} -# maxvalue: -4.0 -{% if letkf_app | default(false) %} - obs localizations: - - localization method: Rossby - base value: 100.0e3 - rossby mult: 1.0 - min grid mult: 2.0 - min value: 200.0e3 - max value: 900.0e3 -{% endif %} diff --git a/parm/jcb-gdas/observations/marine/icec_atms_n21_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_atms_n21_l2.yaml.j2 new file mode 120000 index 000000000..3ebe0973c --- /dev/null +++ b/parm/jcb-gdas/observations/marine/icec_atms_n21_l2.yaml.j2 @@ -0,0 +1 @@ +icec_generic_passive.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/icec_atms_npp_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_atms_npp_l2.yaml.j2 deleted file mode 100644 index 7c6cbdc66..000000000 --- a/parm/jcb-gdas/observations/marine/icec_atms_npp_l2.yaml.j2 +++ /dev/null @@ -1,67 +0,0 @@ -- obs space: - name: {{observation_from_jcb}} - obsdatain: - engine: - type: H5File - obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" - obsdataout: - engine: - type: H5File - obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" - simulated variables: [seaIceFraction] - io pool: - max pool size: 1 - get values: - time interpolation: linear - obs operator: - name: Identity - observation alias file: obsop_name_map.yaml - obs error: - covariance model: diagonal - obs filters: - - filter: Domain Check - where: - - variable: {name: GeoVaLs/sea_area_fraction} - value: is_valid - minvalue: 0.9 - - filter: Bounds Check - minvalue: 0.0 - maxvalue: 1.0 - #- filter: Background Check - # threshold: 5.0 - - filter: Domain Check - where: - - variable: { name: GeoVaLs/sea_surface_temperature} - maxvalue: 2.0 - - filter: Domain Check - where: - - variable: { name: GeoVaLs/sea_surface_temperature} - maxvalue: 0.0 - action: - name: inflate error - inflation factor: 2.0 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 0e3 - - filter: Gaussian Thinning - horizontal_mesh: 25 #km - use_reduced_horizontal_grid: true - select_median: true - action: - name: reduce obs space -# - filter: Domain Check -# action: -# name: passivate -# where: -# - variable: {name: GeoVaLs/sea_surface_temperature} -# maxvalue: -4.0 -{% if letkf_app | default(false) %} - obs localizations: - - localization method: Rossby - base value: 100.0e3 - rossby mult: 1.0 - min grid mult: 2.0 - min value: 200.0e3 - max value: 900.0e3 -{% endif %} diff --git a/parm/jcb-gdas/observations/marine/icec_atms_npp_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_atms_npp_l2.yaml.j2 new file mode 120000 index 000000000..3ebe0973c --- /dev/null +++ b/parm/jcb-gdas/observations/marine/icec_atms_npp_l2.yaml.j2 @@ -0,0 +1 @@ +icec_generic_passive.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/icec_generic_passive.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_generic_passive.yaml.j2 new file mode 100644 index 000000000..54ae75f2b --- /dev/null +++ b/parm/jcb-gdas/observations/marine/icec_generic_passive.yaml.j2 @@ -0,0 +1,67 @@ +- obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" + simulated variables: [seaIceFraction] + io pool: + max pool size: 1 + get values: + time interpolation: linear + obs operator: + name: Identity + observation alias file: obsop_name_map.yaml + obs error: + covariance model: diagonal + obs filters: + - filter: Domain Check + where: + - variable: {name: GeoVaLs/sea_area_fraction} + value: is_valid + minvalue: 0.9 + - filter: Bounds Check + minvalue: 0.0 + maxvalue: 1.0 + #- filter: Background Check + # threshold: 5.0 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_surface_temperature} + maxvalue: 2.0 + - filter: Domain Check + where: + - variable: { name: GeoVaLs/sea_surface_temperature} + maxvalue: 0.0 + action: + name: inflate error + inflation factor: 2.0 + - filter: Domain Check + where: + - variable: {name: GeoVaLs/distance_from_coast} + minvalue: 0e3 + - filter: Gaussian Thinning + horizontal_mesh: 25 #km + use_reduced_horizontal_grid: true + select_median: true + action: + name: reduce obs space + - filter: Domain Check + action: + name: passivate + where: + - variable: {name: GeoVaLs/sea_surface_temperature} + maxvalue: -4.0 +{% if letkf_app | default(false) %} + obs localizations: + - localization method: Rossby + base value: 100.0e3 + rossby mult: 1.0 + min grid mult: 2.0 + min value: 200.0e3 + max value: 900.0e3 +{% endif %} diff --git a/parm/jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 deleted file mode 100644 index 695b22409..000000000 --- a/parm/jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 +++ /dev/null @@ -1,61 +0,0 @@ -- obs space: - name: {{observation_from_jcb}} - obsdatain: - engine: - type: H5File - obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" - obsdataout: - engine: - type: H5File - obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" - simulated variables: [seaIceFraction] - io pool: - max pool size: 1 - get values: - time interpolation: linear - obs operator: - name: Identity - observation alias file: obsop_name_map.yaml - obs error: - covariance model: diagonal - obs filters: - - filter: Domain Check - where: - - variable: {name: GeoVaLs/sea_area_fraction} - value: is_valid - minvalue: 0.9 - - filter: Bounds Check - minvalue: 0.0 - maxvalue: 1.0 - #- filter: Background Check - # threshold: 5.0 - - filter: Domain Check - where: - - variable: { name: GeoVaLs/sea_surface_temperature} - maxvalue: 2.0 - - filter: Domain Check - where: - - variable: { name: GeoVaLs/sea_surface_temperature} - maxvalue: 0.0 - action: - name: inflate error - inflation factor: 2.0 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 0e3 -# - filter: Domain Check -# action: -# name: passivate -# where: -# - variable: {name: GeoVaLs/sea_surface_temperature} -# maxvalue: -4.0 -{% if letkf_app | default(false) %} - obs localizations: - - localization method: Rossby - base value: 100.0e3 - rossby mult: 1.0 - min grid mult: 2.0 - min value: 200.0e3 - max value: 900.0e3 -{% endif %} diff --git a/parm/jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 new file mode 120000 index 000000000..3ebe0973c --- /dev/null +++ b/parm/jcb-gdas/observations/marine/icec_gmi_gpm_l2.yaml.j2 @@ -0,0 +1 @@ +icec_generic_passive.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/icec_viirs_j01_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_viirs_j01_l2.yaml.j2 new file mode 120000 index 000000000..3ebe0973c --- /dev/null +++ b/parm/jcb-gdas/observations/marine/icec_viirs_j01_l2.yaml.j2 @@ -0,0 +1 @@ +icec_generic_passive.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_north.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_north.yaml.j2 deleted file mode 100644 index 54ae75f2b..000000000 --- a/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_north.yaml.j2 +++ /dev/null @@ -1,67 +0,0 @@ -- obs space: - name: {{observation_from_jcb}} - obsdatain: - engine: - type: H5File - obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" - obsdataout: - engine: - type: H5File - obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" - simulated variables: [seaIceFraction] - io pool: - max pool size: 1 - get values: - time interpolation: linear - obs operator: - name: Identity - observation alias file: obsop_name_map.yaml - obs error: - covariance model: diagonal - obs filters: - - filter: Domain Check - where: - - variable: {name: GeoVaLs/sea_area_fraction} - value: is_valid - minvalue: 0.9 - - filter: Bounds Check - minvalue: 0.0 - maxvalue: 1.0 - #- filter: Background Check - # threshold: 5.0 - - filter: Domain Check - where: - - variable: { name: GeoVaLs/sea_surface_temperature} - maxvalue: 2.0 - - filter: Domain Check - where: - - variable: { name: GeoVaLs/sea_surface_temperature} - maxvalue: 0.0 - action: - name: inflate error - inflation factor: 2.0 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 0e3 - - filter: Gaussian Thinning - horizontal_mesh: 25 #km - use_reduced_horizontal_grid: true - select_median: true - action: - name: reduce obs space - - filter: Domain Check - action: - name: passivate - where: - - variable: {name: GeoVaLs/sea_surface_temperature} - maxvalue: -4.0 -{% if letkf_app | default(false) %} - obs localizations: - - localization method: Rossby - base value: 100.0e3 - rossby mult: 1.0 - min grid mult: 2.0 - min value: 200.0e3 - max value: 900.0e3 -{% endif %} diff --git a/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_north.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_north.yaml.j2 new file mode 120000 index 000000000..3ebe0973c --- /dev/null +++ b/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_north.yaml.j2 @@ -0,0 +1 @@ +icec_generic_passive.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_south.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_south.yaml.j2 deleted file mode 100644 index 54ae75f2b..000000000 --- a/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_south.yaml.j2 +++ /dev/null @@ -1,67 +0,0 @@ -- obs space: - name: {{observation_from_jcb}} - obsdatain: - engine: - type: H5File - obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" - obsdataout: - engine: - type: H5File - obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" - simulated variables: [seaIceFraction] - io pool: - max pool size: 1 - get values: - time interpolation: linear - obs operator: - name: Identity - observation alias file: obsop_name_map.yaml - obs error: - covariance model: diagonal - obs filters: - - filter: Domain Check - where: - - variable: {name: GeoVaLs/sea_area_fraction} - value: is_valid - minvalue: 0.9 - - filter: Bounds Check - minvalue: 0.0 - maxvalue: 1.0 - #- filter: Background Check - # threshold: 5.0 - - filter: Domain Check - where: - - variable: { name: GeoVaLs/sea_surface_temperature} - maxvalue: 2.0 - - filter: Domain Check - where: - - variable: { name: GeoVaLs/sea_surface_temperature} - maxvalue: 0.0 - action: - name: inflate error - inflation factor: 2.0 - - filter: Domain Check - where: - - variable: {name: GeoVaLs/distance_from_coast} - minvalue: 0e3 - - filter: Gaussian Thinning - horizontal_mesh: 25 #km - use_reduced_horizontal_grid: true - select_median: true - action: - name: reduce obs space - - filter: Domain Check - action: - name: passivate - where: - - variable: {name: GeoVaLs/sea_surface_temperature} - maxvalue: -4.0 -{% if letkf_app | default(false) %} - obs localizations: - - localization method: Rossby - base value: 100.0e3 - rossby mult: 1.0 - min grid mult: 2.0 - min value: 200.0e3 - max value: 900.0e3 -{% endif %} diff --git a/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_south.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_south.yaml.j2 new file mode 120000 index 000000000..3ebe0973c --- /dev/null +++ b/parm/jcb-gdas/observations/marine/icec_viirs_n20_l2_south.yaml.j2 @@ -0,0 +1 @@ +icec_generic_passive.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/icec_viirs_n21_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_viirs_n21_l2.yaml.j2 new file mode 120000 index 000000000..3ebe0973c --- /dev/null +++ b/parm/jcb-gdas/observations/marine/icec_viirs_n21_l2.yaml.j2 @@ -0,0 +1 @@ +icec_generic_passive.yaml.j2 \ No newline at end of file diff --git a/parm/jcb-gdas/observations/marine/icec_viirs_npp_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_viirs_npp_l2.yaml.j2 new file mode 120000 index 000000000..3ebe0973c --- /dev/null +++ b/parm/jcb-gdas/observations/marine/icec_viirs_npp_l2.yaml.j2 @@ -0,0 +1 @@ +icec_generic_passive.yaml.j2 \ No newline at end of file From 805eafd1115ae3eab1ca3c45b0049f23b7482bab Mon Sep 17 00:00:00 2001 From: Guillaume Vernieres Date: Tue, 30 Sep 2025 10:35:20 -0400 Subject: [PATCH 160/199] Addition of the "missing file action" yaml key to the marine obs yamls (#185) Thanks to @CoryMartin-NOAA for pointing us to a simple fix: - fixes #183 --- parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/icec_generic_passive.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/insitu_profile_glider.yaml.j2 | 1 + .../observations/marine/insitu_profile_marinemammal.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/insitu_profile_tesac.yaml.j2 | 1 + .../observations/marine/insitu_profile_tesac_salinity.yaml.j2 | 1 + .../observations/marine/insitu_profile_tropical.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/insitu_profile_xbtctd.yaml.j2 | 1 + .../observations/marine/insitu_salt_profile_argo.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/insitu_surface_altkob.yaml.j2 | 1 + .../observations/marine/insitu_surface_dbuoyb_drifter.yaml.j2 | 2 +- parm/jcb-gdas/observations/marine/insitu_surface_trkob.yaml.j2 | 1 + .../observations/marine/insitu_surface_trkob_salinity.yaml.j2 | 1 + .../observations/marine/insitu_temp_profile_argo.yaml.j2 | 1 + .../observations/marine/insitu_temp_surface_drifter.yaml.j2 | 2 +- parm/jcb-gdas/observations/marine/sss_smap_l2.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/sss_smos_l2.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 | 1 + parm/jcb-gdas/observations/marine/sst_generic_geo.yaml.j2 | 1 + 23 files changed, 23 insertions(+), 2 deletions(-) diff --git a/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 b/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 index c1db68548..3ab01da90 100644 --- a/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 index 06d647c42..c8d428940 100644 --- a/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 index 06d647c42..c8d428940 100644 --- a/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/marine/icec_generic_passive.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_generic_passive.yaml.j2 index 54ae75f2b..0cb551814 100644 --- a/parm/jcb-gdas/observations/marine/icec_generic_passive.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_generic_passive.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 index 7c6cbdc66..700fdfb6c 100644 --- a/parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 index b622fd06c..ad936b25e 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_glider.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_glider.yaml.j2 index 65a5a704e..124799890 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_glider.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_glider.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_marinemammal.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_marinemammal.yaml.j2 index 65a5a704e..124799890 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_marinemammal.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_marinemammal.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_tesac.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_tesac.yaml.j2 index cc6e98786..a004fefe2 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_tesac.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_tesac.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_tesac_salinity.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_tesac_salinity.yaml.j2 index 1fdc5d262..a92e2bf49 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_tesac_salinity.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_tesac_salinity.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_tropical.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_tropical.yaml.j2 index 238b81762..053187439 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_tropical.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_tropical.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_xbtctd.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_xbtctd.yaml.j2 index 65a5a704e..124799890 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_xbtctd.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_xbtctd.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 index e7e0c0463..91a76c650 100644 --- a/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + missing file action: warn obsgrouping: group variables: [latitude, longitude, dateTime] sort variable: depth diff --git a/parm/jcb-gdas/observations/marine/insitu_surface_altkob.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_surface_altkob.yaml.j2 index cb8c277c6..2192f9958 100644 --- a/parm/jcb-gdas/observations/marine/insitu_surface_altkob.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_surface_altkob.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/marine/insitu_surface_dbuoyb_drifter.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_surface_dbuoyb_drifter.yaml.j2 index d3a1a4a3f..c79cd5b69 100644 --- a/parm/jcb-gdas/observations/marine/insitu_surface_dbuoyb_drifter.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_surface_dbuoyb_drifter.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File @@ -53,4 +54,3 @@ min value: 200.0e3 max value: 900.0e3 {% endif %} - diff --git a/parm/jcb-gdas/observations/marine/insitu_surface_trkob.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_surface_trkob.yaml.j2 index be3805708..6dea2041e 100644 --- a/parm/jcb-gdas/observations/marine/insitu_surface_trkob.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_surface_trkob.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/marine/insitu_surface_trkob_salinity.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_surface_trkob_salinity.yaml.j2 index e162f3bc3..80ca4e854 100644 --- a/parm/jcb-gdas/observations/marine/insitu_surface_trkob_salinity.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_surface_trkob_salinity.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 index 8edb0b136..b193a594b 100644 --- a/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + missing file action: warn obsgrouping: group variables: [latitude, longitude, dateTime] sort variable: depth diff --git a/parm/jcb-gdas/observations/marine/insitu_temp_surface_drifter.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_temp_surface_drifter.yaml.j2 index 87a15170c..2b2830fee 100644 --- a/parm/jcb-gdas/observations/marine/insitu_temp_surface_drifter.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_temp_surface_drifter.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File @@ -57,4 +58,3 @@ min value: 200.0e3 max value: 900.0e3 {% endif %} - diff --git a/parm/jcb-gdas/observations/marine/sss_smap_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/sss_smap_l2.yaml.j2 index bdca422dd..aeaa34517 100644 --- a/parm/jcb-gdas/observations/marine/sss_smap_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sss_smap_l2.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/marine/sss_smos_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/sss_smos_l2.yaml.j2 index bdca422dd..aeaa34517 100644 --- a/parm/jcb-gdas/observations/marine/sss_smos_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sss_smos_l2.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 index 02cd72106..b9fd51d5e 100644 --- a/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/marine/sst_generic_geo.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_generic_geo.yaml.j2 index 68e2b7349..75b148e0d 100644 --- a/parm/jcb-gdas/observations/marine/sst_generic_geo.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sst_generic_geo.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{marine_obsdatain_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File From d08a6a798b3f6d1126d0acd037400244726adbb0 Mon Sep 17 00:00:00 2001 From: AndrewEichmann-NOAA <58948505+AndrewEichmann-NOAA@users.noreply.github.com> Date: Thu, 2 Oct 2025 14:30:26 -0400 Subject: [PATCH 161/199] separate ens vs parametric bkgerr stats (#186) Partially resolves https://github.com/NOAA-EMC/GDASApp/issues/1921 . Needs corresponding PRs in GDASApp and global-workflow to not break marine DA. Tested with same on `C48mx500_hybAOWCD` and `C48mx500_3DVarAOWCDA`, should get a full cycling test --- parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 | 2 +- parm/jcb-gdas/algorithm/marine/soca_ensb.yaml.j2 | 2 +- ...marine_background_error_hybrid_diffusion_diffusion.yaml.j2 | 4 ++-- .../marine/marine_background_error_static_diffusion.yaml.j2 | 4 ++-- 4 files changed, 6 insertions(+), 6 deletions(-) diff --git a/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 index 8369a81af..8be8b26d5 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_diagb.yaml.j2 @@ -19,7 +19,7 @@ background: background error: datadir: ./staticb/ date: '{{ marine_window_middle_iso }}' - exp: bkgerr_stddev + exp: bkgerr_parametric_stddev type: incr variables: diff --git a/parm/jcb-gdas/algorithm/marine/soca_ensb.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_ensb.yaml.j2 index ab437fafa..794dfadc5 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_ensb.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_ensb.yaml.j2 @@ -87,7 +87,7 @@ ssh output: background error output: datadir: ./staticb/ date: '{{ marine_window_begin_iso }}' - exp: bkgerr_stddev + exp: bkgerr_ens_stddev type: incr linear variable change: diff --git a/parm/jcb-gdas/model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 b/parm/jcb-gdas/model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 index 30ff4218f..2833fd422 100644 --- a/parm/jcb-gdas/model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 +++ b/parm/jcb-gdas/model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 @@ -30,8 +30,8 @@ components: model file: date: '{{marine_stddev_time}}' basename: ./staticb/ - ocn_filename: 'ocn.bkgerr_stddev.nc' - ice_filename: 'ice.bkgerr_stddev.nc' + ocn_filename: 'ocn.bkgerr_ens_stddev.nc' + ice_filename: 'ice.bkgerr_ens_stddev.nc' read_from_file: 3 linear variable change: input variables: diff --git a/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 b/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 index 1ac58209f..0d663c57f 100644 --- a/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 +++ b/parm/jcb-gdas/model/marine/marine_background_error_static_diffusion.yaml.j2 @@ -27,8 +27,8 @@ saber outer blocks: model file: date: '{{marine_stddev_time}}' basename: ./staticb/ - ocn_filename: 'ocn.bkgerr_stddev.nc' - ice_filename: 'ice.bkgerr_stddev.nc' + ocn_filename: 'ocn.bkgerr_parametric_stddev.nc' + ice_filename: 'ice.bkgerr_parametric_stddev.nc' read_from_file: 3 #- saber block name: MLBalance From de9f25a0358138736c607de330b58e200c84c43d Mon Sep 17 00:00:00 2001 From: Jiarui Dong Date: Mon, 6 Oct 2025 14:19:46 -0400 Subject: [PATCH 162/199] Add observation YAML and chronicle files for new snow observations (#188) This PR includes new observation YAML files: one for snomad and one for the combined snocvr and snomad. This PR also adds the observation chronicle files for the newly introduced observations. --- .../snow/t00z/snocvr_snomad.yaml | 14 ++ .../snow/t00z/snomad.yaml | 14 ++ .../snow/t06z/snocvr_snomad.yaml | 14 ++ .../snow/t06z/snomad.yaml | 14 ++ .../observations/snow/snocvr_snomad.yaml.j2 | 216 +++++++++++++++++ .../jcb-gdas/observations/snow/snomad.yaml.j2 | 217 ++++++++++++++++++ 6 files changed, 489 insertions(+) create mode 100644 parm/jcb-gdas/observation_chronicle/snow/t00z/snocvr_snomad.yaml create mode 100644 parm/jcb-gdas/observation_chronicle/snow/t00z/snomad.yaml create mode 100644 parm/jcb-gdas/observation_chronicle/snow/t06z/snocvr_snomad.yaml create mode 100644 parm/jcb-gdas/observation_chronicle/snow/t06z/snomad.yaml create mode 100644 parm/jcb-gdas/observations/snow/snocvr_snomad.yaml.j2 create mode 100644 parm/jcb-gdas/observations/snow/snomad.yaml.j2 diff --git a/parm/jcb-gdas/observation_chronicle/snow/t00z/snocvr_snomad.yaml b/parm/jcb-gdas/observation_chronicle/snow/t00z/snocvr_snomad.yaml new file mode 100644 index 000000000..6a249526c --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/snow/t00z/snocvr_snomad.yaml @@ -0,0 +1,14 @@ +# Instrument metadata +# ------------------- +commissioned: 2025-09-01T00:00:00 + +observer_type: conventional # Type of chronicle to use + +window_option: max + +# observation type initial configuration +# -------------------------------- +stations_to_reject: [fakelist] + +# Chronicle of changes for this observation type +# ---------------------------------------- diff --git a/parm/jcb-gdas/observation_chronicle/snow/t00z/snomad.yaml b/parm/jcb-gdas/observation_chronicle/snow/t00z/snomad.yaml new file mode 100644 index 000000000..6a249526c --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/snow/t00z/snomad.yaml @@ -0,0 +1,14 @@ +# Instrument metadata +# ------------------- +commissioned: 2025-09-01T00:00:00 + +observer_type: conventional # Type of chronicle to use + +window_option: max + +# observation type initial configuration +# -------------------------------- +stations_to_reject: [fakelist] + +# Chronicle of changes for this observation type +# ---------------------------------------- diff --git a/parm/jcb-gdas/observation_chronicle/snow/t06z/snocvr_snomad.yaml b/parm/jcb-gdas/observation_chronicle/snow/t06z/snocvr_snomad.yaml new file mode 100644 index 000000000..6a249526c --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/snow/t06z/snocvr_snomad.yaml @@ -0,0 +1,14 @@ +# Instrument metadata +# ------------------- +commissioned: 2025-09-01T00:00:00 + +observer_type: conventional # Type of chronicle to use + +window_option: max + +# observation type initial configuration +# -------------------------------- +stations_to_reject: [fakelist] + +# Chronicle of changes for this observation type +# ---------------------------------------- diff --git a/parm/jcb-gdas/observation_chronicle/snow/t06z/snomad.yaml b/parm/jcb-gdas/observation_chronicle/snow/t06z/snomad.yaml new file mode 100644 index 000000000..6a249526c --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/snow/t06z/snomad.yaml @@ -0,0 +1,14 @@ +# Instrument metadata +# ------------------- +commissioned: 2025-09-01T00:00:00 + +observer_type: conventional # Type of chronicle to use + +window_option: max + +# observation type initial configuration +# -------------------------------- +stations_to_reject: [fakelist] + +# Chronicle of changes for this observation type +# ---------------------------------------- diff --git a/parm/jcb-gdas/observations/snow/snocvr_snomad.yaml.j2 b/parm/jcb-gdas/observations/snow/snocvr_snomad.yaml.j2 new file mode 100644 index 000000000..ade714cfd --- /dev/null +++ b/parm/jcb-gdas/observations/snow/snocvr_snomad.yaml.j2 @@ -0,0 +1,216 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: snocvr_snomad + obsdatain: + engine: + type: H5File + obsfile: "{{snow_prepobs_path}}/{{snow_obsdatain_prefix}}{{observation_from_jcb}}.tm00.nc" + missing file action: warn + obsgrouping: + group variables: [stationIdentification] + obsdataout: + engine: + type: H5File + obsfile: "{{snow_obsdataout_path}}/{{snow_obsdataout_prefix}}{{observation_from_jcb}}{{snow_obsdataout_suffix}}" + simulated variables: [totalSnowDepth] + # + + # Observation Operator + # -------------------- + obs operator: + name: Identity + # + + # Observation Filters (QC) + # ------------------------ + obs pre filters: + - filter: Create Diagnostic Flags + flags: + - name: missing_snowdepth + initial value: false + - name: missing_elevation + initial value: false + - name: temporal_thinning + initial value: false + - name: invalid_snowdepth + initial value: false + - name: invalid_elevation + initial value: false + - name: land_check + initial value: false + - name: landice_check + initial value: false + - name: seaice_check + initial value: false + - name: elevation_bkgdiff + initial value: false + - name: rejectlist + initial value: false + - name: background_check + initial value: false + - filter: Perform Action + filter variables: + - name: totalSnowDepth + action: + name: assign error + error parameter: 40.0 + - filter: Domain Check + where: + - variable: + name: ObsValue/totalSnowDepth + value: is_valid + actions: + - name: set + flag: missing_snowdepth + ignore: rejected observations + - name: reject + - filter: Domain Check + where: + - variable: + name: MetaData/stationElevation + value: is_valid + actions: + - name: set + flag: missing_elevation + ignore: rejected observations + - name: reject + - filter: Temporal Thinning + min_spacing: '{{ window_length }}' + seed_time: '{{ snow_background_time_iso }}' + category_variable: + name: MetaData/stationIdentification + actions: + - name: set + flag: temporal_thinning + ignore: rejected observations + - name: reject + obs prior filters: + - filter: Bounds Check + filter variables: + - name: totalSnowDepth + minvalue: 0.0 + maxvalue: 20000.0 + actions: + - name: set + flag: invalid_snowdepth + ignore: rejected observations + - name: reject + - filter: Domain Check # land only + where: + - variable: + name: GeoVaLs/fraction_of_land + minvalue: 0.5 + actions: + - name: set + flag: land_check + ignore: rejected observations + - name: reject + - filter: Domain Check + where: + - variable: + name: MetaData/stationElevation + minvalue: -200.0 + maxvalue: 9900.0 + actions: + - name: set + flag: invalid_elevation + ignore: rejected observations + - name: reject + - filter: Domain Check # land only, no sea ice + where: + - variable: + name: GeoVaLs/fraction_of_ice + maxvalue: 0.0 + actions: + - name: set + flag: seaice_check + ignore: rejected observations + - name: reject + - filter: RejectList # no land-ice + where: + - variable: + name: GeoVaLs/vtype + minvalue: 14.5 + maxvalue: 15.5 + actions: + - name: set + flag: landice_check + ignore: rejected observations + - name: reject + - filter: Difference Check # elevation check + reference: MetaData/stationElevation + value: GeoVaLs/filtered_orography + threshold: 800. + actions: + - name: set + flag: elevation_bkgdiff + ignore: rejected observations + - name: reject + # Add inflate error characterized by a function of elevation difference + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/elevation_diff + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: [MetaData/stationElevation, GeoVaLs/filtered_orography] + coefficients: [1., -1.] + total exponent: 2 + total coefficient: 0.0000015625 #1/(800*800) + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/elevation_diff_exp + type: float + function: + name: ObsFunction/Exponential + options: + variables: [DerivedMetaData/elevation_diff] + coeffA: 1. + coeffB: -1. + coeffC: 0. + coeffD: 1. + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/elevation_diff_exp_inv + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: [DerivedMetaData/elevation_diff_exp] + total exponent: -1. + - filter: Perform Action + filter variables: + - name: totalSnowDepth + where: + - variable: + name: ObsValue/totalSnowDepth + minvalue: 0. + action: + name: inflate error + inflation variable: DerivedMetaData/elevation_diff_exp_inv # 2.0 + - filter: BlackList + where: + - variable: + name: MetaData/stationIdentification + is_in: {{ get_conventional_rejected_stations(observation_from_jcb) }} + actions: + - name: set + flag: rejectlist + ignore: rejected observations + - name: reject + obs post filters: + - filter: Background Check # gross error check + filter variables: + - name: totalSnowDepth + absolute threshold: 250 + bias correction parameter: 1.0 + actions: + - name: set + flag: background_check + ignore: rejected observations + - name: reject + diff --git a/parm/jcb-gdas/observations/snow/snomad.yaml.j2 b/parm/jcb-gdas/observations/snow/snomad.yaml.j2 new file mode 100644 index 000000000..678d9cf99 --- /dev/null +++ b/parm/jcb-gdas/observations/snow/snomad.yaml.j2 @@ -0,0 +1,217 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: snomad + obsdatain: + engine: + type: bufr + obsfile: "{{snow_obsdatain_path}}/{{snow_obsdatain_prefix}}{{observation_from_jcb}}{{snow_obsdatain_suffix}}" + mapping file: "{{snow_obsdatain_path}}/bufr_snomad_mapping.yaml" + missing file action: warn + obsgrouping: + group variables: [stationIdentification] + obsdataout: + engine: + type: H5File + obsfile: "{{snow_obsdataout_path}}/{{snow_obsdataout_prefix}}{{observation_from_jcb}}{{snow_obsdataout_suffix}}" + simulated variables: [totalSnowDepth] + # + + # Observation Operator + # -------------------- + obs operator: + name: Identity + # + + # Observation Filters (QC) + # ------------------------ + obs pre filters: + - filter: Create Diagnostic Flags + flags: + - name: missing_snowdepth + initial value: false + - name: missing_elevation + initial value: false + - name: temporal_thinning + initial value: false + - name: invalid_snowdepth + initial value: false + - name: invalid_elevation + initial value: false + - name: land_check + initial value: false + - name: landice_check + initial value: false + - name: seaice_check + initial value: false + - name: elevation_bkgdiff + initial value: false + - name: rejectlist + initial value: false + - name: background_check + initial value: false + - filter: Perform Action + filter variables: + - name: totalSnowDepth + action: + name: assign error + error parameter: 40.0 + - filter: Domain Check + where: + - variable: + name: ObsValue/totalSnowDepth + value: is_valid + actions: + - name: set + flag: missing_snowdepth + ignore: rejected observations + - name: reject + - filter: Domain Check + where: + - variable: + name: MetaData/stationElevation + value: is_valid + actions: + - name: set + flag: missing_elevation + ignore: rejected observations + - name: reject + - filter: Temporal Thinning + min_spacing: '{{ window_length }}' + seed_time: '{{ snow_background_time_iso }}' + category_variable: + name: MetaData/stationIdentification + actions: + - name: set + flag: temporal_thinning + ignore: rejected observations + - name: reject + obs prior filters: + - filter: Bounds Check + filter variables: + - name: totalSnowDepth + minvalue: 0.0 + maxvalue: 20000.0 + actions: + - name: set + flag: invalid_snowdepth + ignore: rejected observations + - name: reject + - filter: Domain Check # land only + where: + - variable: + name: GeoVaLs/fraction_of_land + minvalue: 0.5 + actions: + - name: set + flag: land_check + ignore: rejected observations + - name: reject + - filter: Domain Check + where: + - variable: + name: MetaData/stationElevation + minvalue: -200.0 + maxvalue: 9900.0 + actions: + - name: set + flag: invalid_elevation + ignore: rejected observations + - name: reject + - filter: Domain Check # land only, no sea ice + where: + - variable: + name: GeoVaLs/fraction_of_ice + maxvalue: 0.0 + actions: + - name: set + flag: seaice_check + ignore: rejected observations + - name: reject + - filter: RejectList # no land-ice + where: + - variable: + name: GeoVaLs/vtype + minvalue: 14.5 + maxvalue: 15.5 + actions: + - name: set + flag: landice_check + ignore: rejected observations + - name: reject + - filter: Difference Check # elevation check + reference: MetaData/stationElevation + value: GeoVaLs/filtered_orography + threshold: 800. + actions: + - name: set + flag: elevation_bkgdiff + ignore: rejected observations + - name: reject + # Add inflate error characterized by a function of elevation difference + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/elevation_diff + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: [MetaData/stationElevation, GeoVaLs/filtered_orography] + coefficients: [1., -1.] + total exponent: 2 + total coefficient: 0.0000015625 #1/(800*800) + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/elevation_diff_exp + type: float + function: + name: ObsFunction/Exponential + options: + variables: [DerivedMetaData/elevation_diff] + coeffA: 1. + coeffB: -1. + coeffC: 0. + coeffD: 1. + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/elevation_diff_exp_inv + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: [DerivedMetaData/elevation_diff_exp] + total exponent: -1. + - filter: Perform Action + filter variables: + - name: totalSnowDepth + where: + - variable: + name: ObsValue/totalSnowDepth + minvalue: 0. + action: + name: inflate error + inflation variable: DerivedMetaData/elevation_diff_exp_inv # 2.0 + - filter: BlackList + where: + - variable: + name: MetaData/stationIdentification + is_in: {{ get_conventional_rejected_stations(observation_from_jcb) }} + actions: + - name: set + flag: rejectlist + ignore: rejected observations + - name: reject + obs post filters: + - filter: Background Check # gross error check + filter variables: + - name: totalSnowDepth + absolute threshold: 250 + bias correction parameter: 1.0 + actions: + - name: set + flag: background_check + ignore: rejected observations + - name: reject + From 4fa9abadffd2d3e44866348fa5e1ce7a39b9d02d Mon Sep 17 00:00:00 2001 From: Guillaume Vernieres Date: Fri, 10 Oct 2025 13:33:04 -0400 Subject: [PATCH 163/199] Add the "cycle date" key to the stats yaml (#187) ~_Still testing ..._~ Done testing SSS, still problematic/biased retrievals so I kept it passive. --- parm/jcb-gdas/algorithm/marine/soca_diag_stats.yaml.j2 | 1 + 1 file changed, 1 insertion(+) diff --git a/parm/jcb-gdas/algorithm/marine/soca_diag_stats.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_diag_stats.yaml.j2 index cc06fa645..8c774648c 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_diag_stats.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_diag_stats.yaml.j2 @@ -2,6 +2,7 @@ time window: begin: '1900-01-01T00:00:00Z' end: '2035-01-01T00:00:00Z' bound to include: begin +cycle date: {{ marine_window_middle_iso }} obs spaces: {% for observation_from_jcb in cleaned_observations %} - obs space: From e61f2f8a8455c8aaabe73b832de2ef40313d97db Mon Sep 17 00:00:00 2001 From: emilyhcliu <36091766+emilyhcliu@users.noreply.github.com> Date: Wed, 15 Oct 2025 16:46:09 -0400 Subject: [PATCH 164/199] Update comment from 'cloudFree' to 'Data Thinning' (#193) Update incorrect comment. This PR does not change results --- parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/iasi_metop-c.yaml.j2 | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 index 5454945a4..15e1f0ff0 100644 --- a/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 @@ -132,7 +132,7 @@ - name: ObsValue/cloudFree coefs: [100] - # Step 6: Create cloudFree variable + # Step 6: Data Thinning - filter: Gaussian Thinning filter variables: - name: brightnessTemperature diff --git a/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 index 733b71658..8d869ca5b 100644 --- a/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 @@ -132,7 +132,7 @@ - name: ObsValue/cloudFree coefs: [100] - # Step 6: Create cloudFree variable + # Step 6: Data Thinning - filter: Gaussian Thinning filter variables: - name: brightnessTemperature diff --git a/parm/jcb-gdas/observations/atmosphere/iasi_metop-c.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/iasi_metop-c.yaml.j2 index 9565aac22..ced043808 100644 --- a/parm/jcb-gdas/observations/atmosphere/iasi_metop-c.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/iasi_metop-c.yaml.j2 @@ -132,7 +132,7 @@ - name: ObsValue/cloudFree coefs: [100] - # Step 6: Create cloudFree variable + # Step 6: Data Thinning - filter: Gaussian Thinning filter variables: - name: brightnessTemperature From c5f47b14a2b269d27ed8ddbf57c8529afaa6b092 Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Thu, 16 Oct 2025 10:19:11 -0400 Subject: [PATCH 165/199] GSI surface pressure stats template (#191) Add a ioda-stats Jinja2 template for GSI ncdiag surface pressure --- .../atmos_gsi/gsi_sfc_ps_template.yaml.j2 | 49 +++++++++++++++++++ 1 file changed, 49 insertions(+) create mode 100644 parm/jcb-gdas/algorithm/obstats/atmos_gsi/gsi_sfc_ps_template.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/obstats/atmos_gsi/gsi_sfc_ps_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/atmos_gsi/gsi_sfc_ps_template.yaml.j2 new file mode 100644 index 000000000..4f3117ee7 --- /dev/null +++ b/parm/jcb-gdas/algorithm/obstats/atmos_gsi/gsi_sfc_ps_template.yaml.j2 @@ -0,0 +1,49 @@ +- obs space: + name: gsi_sfc_ps + obsdatain: + engine: + type: H5File + obsfile: {{ atmosphere_obsdatain_path }}/atmos_gsi/sfc_ps_gsi_{{ stat_current_cycle_YMDH }}.nc4 + simulated variables: ['stationPressure'] + observed variables: ['stationPressure'] + statistics to compute: ['mean', 'count', 'RMS'] + variables: ['stationPressure'] + groups to process: ['GsiHofXBcGes', 'GsiHofXBcAnl', 'ObsValue'] + qc groups: ['GsiEffectiveQCGes', 'GsiEffectiveQCAnl', 'GsiEffectiveQCGes'] + output file: "sfc_ps_{{ stat_current_cycle_YMDH }}_output_atmos_gsi.nc" + regular grid binning: + bin size in degrees: 1.0 + use negative longitudes: true + domains to process: + - domain: + name: "SH" + first mask variable: latitude + first mask range: [-90,0] + - domain: + name: "NH" + first mask variable: latitude + first mask range: [0,90] + - domain: + name: "CONUS" + first mask variable: latitude + first mask range: [25,49] + second mask variable: longitude + second mask range: [-125,-66] + - domain: + name: "Europe" + first mask variable: latitude + first mask range: [35,70] + second mask variable: longitude + second mask range: [-11,38] + - domain: + name: "Africa" + first mask variable: latitude + first mask range: [-35,37] + second mask variable: longitude + second mask range: [-17,52] + - domain: + name: "Asia" + first mask variable: latitude + first mask range: [0,70] + second mask variable: longitude + second mask range: [38, 180] From 7f54c93a5ff55b54b521353868c12b6eb59e5716 Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Tue, 21 Oct 2025 13:27:50 -0400 Subject: [PATCH 166/199] Use station pressure from 3 sources (#190) This adds station pressure YAMLs for ADPUPA, ADPSFC and SFCSHP --- .../atmos/prepbufr_adpsfc_template.yaml.j2 | 53 +++ ...ml.j2 => prepbufr_adpupa_template.yaml.j2} | 8 +- .../atmos/prepbufr_sfcshp_template.yaml.j2 | 53 +++ .../atmos_gsi/gsi_sfc_ps_template.yaml.j2 | 10 +- ...nal_ps.yaml.j2 => prepbufr_adpsfc.yaml.j2} | 0 .../atmosphere/prepbufr_adpupa.yaml.j2 | 368 ++++++++++++++++++ .../atmosphere/prepbufr_sfcshp.yaml.j2 | 368 ++++++++++++++++++ 7 files changed, 855 insertions(+), 5 deletions(-) create mode 100644 parm/jcb-gdas/algorithm/obstats/atmos/prepbufr_adpsfc_template.yaml.j2 rename parm/jcb-gdas/algorithm/obstats/atmos/{conventional_ps_template.yaml.j2 => prepbufr_adpupa_template.yaml.j2} (81%) create mode 100644 parm/jcb-gdas/algorithm/obstats/atmos/prepbufr_sfcshp_template.yaml.j2 rename parm/jcb-gdas/observations/atmosphere/{conventional_ps.yaml.j2 => prepbufr_adpsfc.yaml.j2} (100%) create mode 100644 parm/jcb-gdas/observations/atmosphere/prepbufr_adpupa.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/prepbufr_sfcshp.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/obstats/atmos/prepbufr_adpsfc_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/atmos/prepbufr_adpsfc_template.yaml.j2 new file mode 100644 index 000000000..51aaca9de --- /dev/null +++ b/parm/jcb-gdas/algorithm/obstats/atmos/prepbufr_adpsfc_template.yaml.j2 @@ -0,0 +1,53 @@ +- obs space: + name: prepbufr_adpsfc + obsdatain: + engine: + type: H5File + obsfile: {{ atmosphere_obsdatain_path }}/atmos/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc + simulated variables: ['stationPressure'] + observed variables: ['stationPressure'] + statistics to compute: ['mean', 'count', 'RMS'] + variables: ['stationPressure'] + groups to process: ['ombg', 'oman'] + qc groups: ['EffectiveQC0', 'EffectiveQC1'] + output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_atmos.nc" + ascii vertical bins: + vertical bin names: [1000+, 1000-900, 900-800, 800-600, 600-400, 400-300, 300-250, 250-200, 200-150, 150-100, 100-50] + vertical bin ranges: [100000, 90000, 80000, 60000, 40000, 30000, 25000, 20000, 15000, 10000, 5000] + vertical bin variable: pressure + regular grid binning: + bin size in degrees: 2.5 + use negative longitudes: true + domains to process: + - domain: + name: "SH" + first mask variable: latitude + first mask range: [-90,0] + - domain: + name: "NH" + first mask variable: latitude + first mask range: [0,90] + - domain: + name: "CONUS" + first mask variable: latitude + first mask range: [25,49] + second mask variable: longitude + second mask range: [-125,-66] + - domain: + name: "Europe" + first mask variable: latitude + first mask range: [35,70] + second mask variable: longitude + second mask range: [-11,38] + - domain: + name: "Africa" + first mask variable: latitude + first mask range: [-35,37] + second mask variable: longitude + second mask range: [-17,52] + - domain: + name: "Asia" + first mask variable: latitude + first mask range: [0,70] + second mask variable: longitude + second mask range: [38, 180] diff --git a/parm/jcb-gdas/algorithm/obstats/atmos/conventional_ps_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/atmos/prepbufr_adpupa_template.yaml.j2 similarity index 81% rename from parm/jcb-gdas/algorithm/obstats/atmos/conventional_ps_template.yaml.j2 rename to parm/jcb-gdas/algorithm/obstats/atmos/prepbufr_adpupa_template.yaml.j2 index 57e77fc28..a951d1e27 100644 --- a/parm/jcb-gdas/algorithm/obstats/atmos/conventional_ps_template.yaml.j2 +++ b/parm/jcb-gdas/algorithm/obstats/atmos/prepbufr_adpupa_template.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: prepbufr_adpsfc + name: prepbufr_adpupa obsdatain: engine: type: H5File @@ -11,8 +11,12 @@ groups to process: ['ombg', 'oman'] qc groups: ['EffectiveQC0', 'EffectiveQC1'] output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_atmos.nc" + ascii vertical bins: + vertical bin names: [1000+, 1000-900, 900-800, 800-600, 600-400, 400-300, 300-250, 250-200, 200-150, 150-100, 100-50] + vertical bin ranges: [100000, 90000, 80000, 60000, 40000, 30000, 25000, 20000, 15000, 10000, 5000] + vertical bin variable: pressure regular grid binning: - bin size in degrees: 1.0 + bin size in degrees: 2.5 use negative longitudes: true domains to process: - domain: diff --git a/parm/jcb-gdas/algorithm/obstats/atmos/prepbufr_sfcshp_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/atmos/prepbufr_sfcshp_template.yaml.j2 new file mode 100644 index 000000000..58539ad6f --- /dev/null +++ b/parm/jcb-gdas/algorithm/obstats/atmos/prepbufr_sfcshp_template.yaml.j2 @@ -0,0 +1,53 @@ +- obs space: + name: prepbufr_sfcshp + obsdatain: + engine: + type: H5File + obsfile: {{ atmosphere_obsdatain_path }}/atmos/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc + simulated variables: ['stationPressure'] + observed variables: ['stationPressure'] + statistics to compute: ['mean', 'count', 'RMS'] + variables: ['stationPressure'] + groups to process: ['ombg', 'oman'] + qc groups: ['EffectiveQC0', 'EffectiveQC1'] + output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_atmos.nc" + ascii vertical bins: + vertical bin names: [1000+, 1000-900, 900-800, 800-600, 600-400, 400-300, 300-250, 250-200, 200-150, 150-100, 100-50] + vertical bin ranges: [100000, 90000, 80000, 60000, 40000, 30000, 25000, 20000, 15000, 10000, 5000] + vertical bin variable: pressure + regular grid binning: + bin size in degrees: 2.5 + use negative longitudes: true + domains to process: + - domain: + name: "SH" + first mask variable: latitude + first mask range: [-90,0] + - domain: + name: "NH" + first mask variable: latitude + first mask range: [0,90] + - domain: + name: "CONUS" + first mask variable: latitude + first mask range: [25,49] + second mask variable: longitude + second mask range: [-125,-66] + - domain: + name: "Europe" + first mask variable: latitude + first mask range: [35,70] + second mask variable: longitude + second mask range: [-11,38] + - domain: + name: "Africa" + first mask variable: latitude + first mask range: [-35,37] + second mask variable: longitude + second mask range: [-17,52] + - domain: + name: "Asia" + first mask variable: latitude + first mask range: [0,70] + second mask variable: longitude + second mask range: [38, 180] diff --git a/parm/jcb-gdas/algorithm/obstats/atmos_gsi/gsi_sfc_ps_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/atmos_gsi/gsi_sfc_ps_template.yaml.j2 index 4f3117ee7..568ee9aeb 100644 --- a/parm/jcb-gdas/algorithm/obstats/atmos_gsi/gsi_sfc_ps_template.yaml.j2 +++ b/parm/jcb-gdas/algorithm/obstats/atmos_gsi/gsi_sfc_ps_template.yaml.j2 @@ -8,11 +8,15 @@ observed variables: ['stationPressure'] statistics to compute: ['mean', 'count', 'RMS'] variables: ['stationPressure'] - groups to process: ['GsiHofXBcGes', 'GsiHofXBcAnl', 'ObsValue'] - qc groups: ['GsiEffectiveQCGes', 'GsiEffectiveQCAnl', 'GsiEffectiveQCGes'] + groups to process: ['ombg', 'oman'] + qc groups: ['GsiEffectiveQCGes', 'GsiEffectiveQCAnl'] output file: "sfc_ps_{{ stat_current_cycle_YMDH }}_output_atmos_gsi.nc" + ascii vertical bins: + vertical bin names: [1000+, 1000-900, 900-800, 800-600, 600-400, 400-300, 300-250, 250-200, 200-150, 150-100, 100-50] + vertical bin ranges: [100000, 90000, 80000, 60000, 40000, 30000, 25000, 20000, 15000, 10000, 5000] + vertical bin variable: pressure regular grid binning: - bin size in degrees: 1.0 + bin size in degrees: 2.5 use negative longitudes: true domains to process: - domain: diff --git a/parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/prepbufr_adpsfc.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/prepbufr_adpsfc.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/prepbufr_adpupa.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/prepbufr_adpupa.yaml.j2 new file mode 100644 index 000000000..ad79038c0 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/prepbufr_adpupa.yaml.j2 @@ -0,0 +1,368 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: surface_ps + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [stationPressure] + + # Observation Operator + # -------------------- + obs operator: + name: SfcCorrected + variables: + - name: stationPressure + correction scheme to use: GSL + station_altitude: height + geovar_sfc_geomz: height_above_mean_sea_level_at_surface + geovar_geomz: geopotential_height + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: Identity + variables: + - name: stationPressure + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Initial Error Assignments for SFC Observations + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [181] + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + round_to_the_nearest_integer: true + xvar: + name: ObsValue/stationPressure + xvals: [80000, 75000, 70000, 65000, 60000, 55000] + errors: [110, 120, 120, 120, 120, 1.0e+11] + + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [187] + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + round_to_the_nearest_integer: true + xvar: + name: ObsValue/stationPressure + xvals: [85000, 80000, 75000, 70000, 65000, 60000, 55000] + errors: [120, 140, 140, 140, 140, 140, 1.0e+11] + + # Initial Error Assignments for SFCSHIP Observations + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [180] + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + round_to_the_nearest_integer: true + xvar: + name: ObsValue/stationPressure + xvals: [60000, 55000] + errors: [130, 1.0e+11] + + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [183] + action: + name: assign error + error parameter: 1.0e+11 + + # Initial Error Assignments for Radiosonde + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [120] + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + round_to_the_nearest_integer: true + xvar: + name: ObsValue/stationPressure + xvals: [80000, 75000, 70000, 65000, 60000, 55000] + errors: [110, 120, 120, 120, 120, 1.0e+11] + + # Create PreQC group variable (pqm in GSI read_prepbufr) + - filter: Variable Assignment + assignments: + - name: InputObsError/stationPressure + type: float + source variable: ObsErrorData/stationPressure + + # Set observation quality-realted variables + # Create PreQC group variable (pqm in GSI read_prepbufr) + - filter: Variable Assignment + assignments: + - name: PreQC/stationPressure + type: int + source variable: QualityMarker/stationPressure + + # Create PreUseFlag group variable (usage in GSI read_prepbufr) + # Initialize + - filter: Variable Assignment + assignments: + - name: PreUseFlag/stationPressure + type: int + source variable: PreQC/stationPressure + + - filter: Variable Assignment + where: + - variable: + name: PreUseFlag/stationPressure + is_in: 1-15 + assignments: + - name: PreUseFlag/stationPressure + value: 0 + # Re-assignment + - filter: Variable Assignment + where: + - variable: + name: ObsType/stationPressure + is_in: 183 + assignments: + - name: PreUseFlag/stationPressure + value: 100 + + - filter: Variable Assignment + where: + - variable: + name: ObsValue/stationPressure + is_defined: + - variable: + name: ObsValue/stationPressure + maxvalue: 50000.00 + where operator: and + assignments: + - name: PreUseFlag/stationPressure + value: 100 + + - filter: Variable Assignment + where: + - variable: + name: PreQC/stationPressure + is_in: 9, 12, 15 + assignments: + - name: PreUseFlag/stationPressure + value: 100 + + - filter: Variable Assignment + where: + - variable: + name: PreQC/stationPressure + is_in: 4-15 + assignments: + - name: PreUseFlag/stationPressure + value: 101 + + ## Observation range sanity check + #- filter: Bounds Check + # filter variables: + # - name: stationPressure + # minvalue: 37499.0 + # maxvalue: 106999.0 + # action: + # name: reject + ## Reject all ObsType 183 + #- filter: RejectList + # where: + # - variable: + # name: ObsType/stationPressure + # is_in: 183 + ## Reject surface pressure below 500 hPa + #- filter: Bounds Check + # filter variables: + # - name: stationPressure + # minvalue: 50000.00 + # action: + # name: reject + #- filter: RejectList + # where: + # - variable: + # name: PreQC/stationPressure + # is_in: 4-15 + # Inflate obs error based on obs type + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: PreQC/stationPressure + is_in: 3, 7 + action: + name: inflate error + inflation factor: 1.2 + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Calculate obs error inflation factors for duplicated observations at the same location + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorDuplicateCheck/stationPressure + type: float + function: + name: ObsFunction/ObsErrorFactorDuplicateCheck + options: + use_air_pressure: false + variable: stationPressure + + # Reduce effective observation error based on obs type and subtype + # In this case: reduce effective obs error for buoy + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: + name: ObsType/stationPressure + is_in: 180 + - variable: + name: ObsSubType/stationPressure + is_in: 0 + action: + name: inflate error + inflation factor: 0.7 + + # Calculate obs error inflation factors for large discrepancies between model and observations + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorSfcPressure/stationPressure + type: float + function: + name: ObsFunction/ObsErrorFactorSfcPressure + options: + geovar_sfc_geomz: height_above_mean_sea_level_at_surface + geovar_geomz: geopotential_height + station_altitude: height + + # Inflate surface pressure observation based on discrepancies between + # model and observations due to terrian + - filter: Perform Action + filter variables: + - name: stationPressure + action: + name: inflate error + inflation variable: + name: ObsErrorFactorSfcPressure/stationPressure + + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/Innovation + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsValue/stationPressure + - name: HofX/stationPressure + coefs: [1, -1] + + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ObsErrorBoundSfcPressure1 + type: float + function: + name: ObsFunction/ObsErrorBoundConventional + options: + obsvar: stationPressure + obserr_bound_min: 100 + obserr_bound_max: 300 + obserr_bound_factor: 5.0 + + - filter: Background Check + filter variables: + - name: stationPressure + where: + - variable: PreQC/stationPressure + is_not_in: 3 + function absolute threshold: + - name: DerivedMetaData/ObsErrorBoundSfcPressure1 + action: + name: reject + + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ObsErrorBoundSfcPressure2 + type: float + function: + name: ObsFunction/ObsErrorBoundConventional + options: + obsvar: stationPressure + obserr_bound_min: 100 + obserr_bound_max: 300 + obserr_bound_factor: 3.5 + + - filter: Background Check + filter variables: + - name: stationPressure + where: + - variable: PreQC/stationPressure + is_in: 3 + function absolute threshold: + - name: DerivedMetaData/ObsErrorBoundSfcPressure2 + action: + name: reject + + # Inflate obs error based on duplicate check + - filter: Perform Action + filter variables: + - name: stationPressure + action: + name: inflate error + inflation variable: + name: ObsErrorFactorDuplicateCheck/stationPressure + + # Reject data based on PreUseFlag (usage in GSI) + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: PreUseFlag/stationPressure + is_not_in: 0, 1 + action: + name: reject + # End of Filters + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/prepbufr_sfcshp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/prepbufr_sfcshp.yaml.j2 new file mode 100644 index 000000000..39ddc387d --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/prepbufr_sfcshp.yaml.j2 @@ -0,0 +1,368 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: surface_ps + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [stationPressure] + + # Observation Operator + # -------------------- + obs operator: + name: SfcCorrected + variables: + - name: stationPressure + correction scheme to use: GSL + station_altitude: height + geovar_sfc_geomz: height_above_mean_sea_level_at_surface + geovar_geomz: geopotential_height + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: Identity + variables: + - name: stationPressure + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Initial Error Assignments for SFC Observations + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [181] + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + round_to_the_nearest_integer: true + xvar: + name: ObsValue/stationPressure + xvals: [80000, 75000, 70000, 65000, 60000, 55000] + errors: [110, 120, 120, 120, 120, 1.0e+11] + + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [187] + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + round_to_the_nearest_integer: true + xvar: + name: ObsValue/stationPressure + xvals: [85000, 80000, 75000, 70000, 65000, 60000, 55000] + errors: [120, 140, 140, 140, 140, 140, 1.0e+11] + + # Initial Error Assignments for SFCSHIP Observations + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [180] + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + round_to_the_nearest_integer: true + xvar: + name: ObsValue/stationPressure + xvals: [60000, 55000] + errors: [130, 1.0e+11] + + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [183] + action: + name: assign error + error parameter: 1.0e+11 + + # Initial Error Assignments for Radiosonde + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [120] + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + round_to_the_nearest_integer: true + xvar: + name: ObsValue/stationPressure + xvals: [80000, 75000, 70000, 65000, 60000, 55000] + errors: [110, 120, 120, 120, 120, 1.0e+11] + + # Create PreQC group variable (pqm in GSI read_prepbufr) + - filter: Variable Assignment + assignments: + - name: InputObsError/stationPressure + type: float + source variable: ObsErrorData/stationPressure + + # Set observation quality-realted variables + # Create PreQC group variable (pqm in GSI read_prepbufr) + - filter: Variable Assignment + assignments: + - name: PreQC/stationPressure + type: int + source variable: QualityMarker/stationPressure + + # Create PreUseFlag group variable (usage in GSI read_prepbufr) + # Initialize + - filter: Variable Assignment + assignments: + - name: PreUseFlag/stationPressure + type: int + source variable: PreQC/stationPressure + + - filter: Variable Assignment + where: + - variable: + name: PreUseFlag/stationPressure + is_in: 1-15 + assignments: + - name: PreUseFlag/stationPressure + value: 0 + # Re-assignment + - filter: Variable Assignment + where: + - variable: + name: ObsType/stationPressure + is_in: 183 + assignments: + - name: PreUseFlag/stationPressure + value: 100 + + - filter: Variable Assignment + where: + - variable: + name: ObsValue/stationPressure + is_defined: + - variable: + name: ObsValue/stationPressure + maxvalue: 50000.00 + where operator: and + assignments: + - name: PreUseFlag/stationPressure + value: 100 + + - filter: Variable Assignment + where: + - variable: + name: PreQC/stationPressure + is_in: 9, 12, 15 + assignments: + - name: PreUseFlag/stationPressure + value: 100 + + - filter: Variable Assignment + where: + - variable: + name: PreQC/stationPressure + is_in: 4-15 + assignments: + - name: PreUseFlag/stationPressure + value: 101 + + ## Observation range sanity check + #- filter: Bounds Check + # filter variables: + # - name: stationPressure + # minvalue: 37499.0 + # maxvalue: 106999.0 + # action: + # name: reject + ## Reject all ObsType 183 + #- filter: RejectList + # where: + # - variable: + # name: ObsType/stationPressure + # is_in: 183 + ## Reject surface pressure below 500 hPa + #- filter: Bounds Check + # filter variables: + # - name: stationPressure + # minvalue: 50000.00 + # action: + # name: reject + #- filter: RejectList + # where: + # - variable: + # name: PreQC/stationPressure + # is_in: 4-15 + # Inflate obs error based on obs type + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: PreQC/stationPressure + is_in: 3, 7 + action: + name: inflate error + inflation factor: 1.2 + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Calculate obs error inflation factors for duplicated observations at the same location + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorDuplicateCheck/stationPressure + type: float + function: + name: ObsFunction/ObsErrorFactorDuplicateCheck + options: + use_air_pressure: false + variable: stationPressure + + # Reduce effective observation error based on obs type and subtype + # In this case: reduce effective obs error for buoy + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: + name: ObsType/stationPressure + is_in: 180 + - variable: + ObsSubType/stationPressure + is_in: 0 + action: + name: inflate error + inflation factor: 0.7 + + # Calculate obs error inflation factors for large discrepancies between model and observations + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorSfcPressure/stationPressure + type: float + function: + name: ObsFunction/ObsErrorFactorSfcPressure + options: + geovar_sfc_geomz: height_above_mean_sea_level_at_surface + geovar_geomz: geopotential_height + station_altitude: height + + # Inflate surface pressure observation based on discrepancies between + # model and observations due to terrian + - filter: Perform Action + filter variables: + - name: stationPressure + action: + name: inflate error + inflation variable: + name: ObsErrorFactorSfcPressure/stationPressure + + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/Innovation + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsValue/stationPressure + - name: HofX/stationPressure + coefs: [1, -1] + + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ObsErrorBoundSfcPressure1 + type: float + function: + name: ObsFunction/ObsErrorBoundConventional + options: + obsvar: stationPressure + obserr_bound_min: 100 + obserr_bound_max: 300 + obserr_bound_factor: 5.0 + + - filter: Background Check + filter variables: + - name: stationPressure + where: + - variable: PreQC/stationPressure + is_not_in: 3 + function absolute threshold: + - name: DerivedMetaData/ObsErrorBoundSfcPressure1 + action: + name: reject + + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ObsErrorBoundSfcPressure2 + type: float + function: + name: ObsFunction/ObsErrorBoundConventional + options: + obsvar: stationPressure + obserr_bound_min: 100 + obserr_bound_max: 300 + obserr_bound_factor: 3.5 + + - filter: Background Check + filter variables: + - name: stationPressure + where: + - variable: PreQC/stationPressure + is_in: 3 + function absolute threshold: + - name: DerivedMetaData/ObsErrorBoundSfcPressure2 + action: + name: reject + + # Inflate obs error based on duplicate check + - filter: Perform Action + filter variables: + - name: stationPressure + action: + name: inflate error + inflation variable: + name: ObsErrorFactorDuplicateCheck/stationPressure + + # Reject data based on PreUseFlag (usage in GSI) + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: PreUseFlag/stationPressure + is_not_in: 0, 1 + action: + name: reject + # End of Filters + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 From d15cbfb847f53c1bb731f0dd06cd2ada6868d467 Mon Sep 17 00:00:00 2001 From: DavidNew-NOAA <134300700+DavidNew-NOAA@users.noreply.github.com> Date: Wed, 22 Oct 2025 14:15:36 -0400 Subject: [PATCH 167/199] JCB companion to Global Workflow marine refactoring (#194) This PR, and companion to https://github.com/NOAA-EMC/GDASApp/pull/1944 and https://github.com/NOAA-EMC/global-workflow/pull/4160, does a few things: 1. It adds templates required for rendering the JEDI marine LETKF `local_ensemble_da` algorithm. 2. It adds a jinja switch to the marine obs YAMLs to turn on a halo distribution when running marine LETKF. 3. It adds the `marine_` prefix to two or three JCB variables. --- .../algorithm/marine/soca_ensb.yaml.j2 | 4 ++-- .../marine/marine_background_ensemble.yaml.j2 | 10 ++++++++ .../marine/marine_output_increment.yaml.j2 | 4 ++++ .../marine/marine_output_mean_prior.yaml.j2 | 4 ++++ .../marine_output_variance_posterior.yaml.j2 | 4 ++++ .../marine_output_variance_prior.yaml.j2 | 4 ++++ .../marine/marine_posterior_output.yaml.j2 | 4 ++++ .../observations/marine/adt_rads_all.yaml.j2 | 7 +++++- .../marine/icec_amsr2_north.yaml.j2 | 23 +++++++++++-------- .../marine/icec_amsr2_south.yaml.j2 | 23 +++++++++++-------- .../marine/icec_generic_passive.yaml.j2 | 7 +++++- .../marine/icec_ssmis_f17_l2.yaml.j2 | 7 +++++- .../marine/insitu_profile_bathy.yaml.j2 | 7 +++++- .../marine/insitu_profile_glider.yaml.j2 | 7 +++++- .../insitu_profile_marinemammal.yaml.j2 | 7 +++++- .../marine/insitu_profile_tesac.yaml.j2 | 7 +++++- .../insitu_profile_tesac_salinity.yaml.j2 | 7 +++++- .../marine/insitu_profile_tropical.yaml.j2 | 7 +++++- .../marine/insitu_profile_xbtctd.yaml.j2 | 7 +++++- .../marine/insitu_salt_profile_argo.yaml.j2 | 7 +++++- .../marine/insitu_surface_altkob.yaml.j2 | 7 +++++- .../insitu_surface_dbuoyb_drifter.yaml.j2 | 7 +++++- .../marine/insitu_surface_trkob.yaml.j2 | 7 +++++- .../insitu_surface_trkob_salinity.yaml.j2 | 7 +++++- .../marine/insitu_temp_profile_argo.yaml.j2 | 7 +++++- .../insitu_temp_surface_drifter.yaml.j2 | 7 +++++- .../observations/marine/sss_smap_l2.yaml.j2 | 7 +++++- .../observations/marine/sss_smos_l2.yaml.j2 | 7 +++++- .../observations/marine/sst_generic.yaml.j2 | 7 +++++- .../marine/sst_generic_geo.yaml.j2 | 7 +++++- 30 files changed, 186 insertions(+), 41 deletions(-) create mode 100644 parm/jcb-gdas/model/marine/marine_background_ensemble.yaml.j2 create mode 100644 parm/jcb-gdas/model/marine/marine_output_increment.yaml.j2 create mode 100644 parm/jcb-gdas/model/marine/marine_output_mean_prior.yaml.j2 create mode 100644 parm/jcb-gdas/model/marine/marine_output_variance_posterior.yaml.j2 create mode 100644 parm/jcb-gdas/model/marine/marine_output_variance_prior.yaml.j2 create mode 100644 parm/jcb-gdas/model/marine/marine_posterior_output.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/marine/soca_ensb.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_ensb.yaml.j2 index 794dfadc5..9351877c7 100644 --- a/parm/jcb-gdas/algorithm/marine/soca_ensb.yaml.j2 +++ b/parm/jcb-gdas/algorithm/marine/soca_ensb.yaml.j2 @@ -38,7 +38,7 @@ soca increments: # Could also be states, but they are read as increments pattern: '%mem%' template: date: '{{ marine_window_begin_iso }}' - basename: '{{ enspert_relpath }}/ens/' + basename: '{{ marine_enspert_relpath }}/ens/' ocn_filename: 'ocean.%mem%.nc' ice_filename: 'ice.%mem%.nc' read_from_file: 3 @@ -114,7 +114,7 @@ trajectory: read_from_file: 1 output increment: - datadir: '{{ enspert_relpath }}/' + datadir: '{{ marine_enspert_relpath }}/' date: '{{ marine_window_begin_iso }}' exp: trash type: incr diff --git a/parm/jcb-gdas/model/marine/marine_background_ensemble.yaml.j2 b/parm/jcb-gdas/model/marine/marine_background_ensemble.yaml.j2 new file mode 100644 index 000000000..911cf80ac --- /dev/null +++ b/parm/jcb-gdas/model/marine/marine_background_ensemble.yaml.j2 @@ -0,0 +1,10 @@ +members from template: + template: + date: '{{ marine_window_middle_iso }}' + ocn_filename: "ocean.%mem%.nc" + ice_filename: "ice.%mem%.nc" + read_from_file: 1 + basename: {{ marine_enspert_relpath }}/ens/ + state variables: [sea_water_salinity, sea_water_potential_temperature, sea_surface_height_above_geoid, sea_water_cell_thickness, eastward_sea_water_velocity, northward_sea_water_velocity, sea_ice_area_fraction] + pattern: '%mem%' + nmembers: {{ marine_number_ensemble_members }} diff --git a/parm/jcb-gdas/model/marine/marine_output_increment.yaml.j2 b/parm/jcb-gdas/model/marine/marine_output_increment.yaml.j2 new file mode 100644 index 000000000..4cae31926 --- /dev/null +++ b/parm/jcb-gdas/model/marine/marine_output_increment.yaml.j2 @@ -0,0 +1,4 @@ +datadir: letkf_output/ +date: '{{ marine_window_begin_iso }}' +exp: letkf.inc +type: ens diff --git a/parm/jcb-gdas/model/marine/marine_output_mean_prior.yaml.j2 b/parm/jcb-gdas/model/marine/marine_output_mean_prior.yaml.j2 new file mode 100644 index 000000000..20fedd4c5 --- /dev/null +++ b/parm/jcb-gdas/model/marine/marine_output_mean_prior.yaml.j2 @@ -0,0 +1,4 @@ +datadir: letkf_output/ +date: '{{ marine_window_begin_iso }}' +exp: letkf.mean_prior +type: fc diff --git a/parm/jcb-gdas/model/marine/marine_output_variance_posterior.yaml.j2 b/parm/jcb-gdas/model/marine/marine_output_variance_posterior.yaml.j2 new file mode 100644 index 000000000..734344d98 --- /dev/null +++ b/parm/jcb-gdas/model/marine/marine_output_variance_posterior.yaml.j2 @@ -0,0 +1,4 @@ +datadir: letkf_output/ +date: '{{ marine_window_begin_iso }}' +exp: letkf.var_post +type: an diff --git a/parm/jcb-gdas/model/marine/marine_output_variance_prior.yaml.j2 b/parm/jcb-gdas/model/marine/marine_output_variance_prior.yaml.j2 new file mode 100644 index 000000000..de97f917c --- /dev/null +++ b/parm/jcb-gdas/model/marine/marine_output_variance_prior.yaml.j2 @@ -0,0 +1,4 @@ +datadir: letkf_output/ +date: '{{ marine_window_begin_iso }}' +exp: letkf.var_prior +type: fc diff --git a/parm/jcb-gdas/model/marine/marine_posterior_output.yaml.j2 b/parm/jcb-gdas/model/marine/marine_posterior_output.yaml.j2 new file mode 100644 index 000000000..fc823ea8d --- /dev/null +++ b/parm/jcb-gdas/model/marine/marine_posterior_output.yaml.j2 @@ -0,0 +1,4 @@ +datadir: letkf_output/ +date: '{{ marine_window_begin_iso }}' +exp: letkf +type: ens diff --git a/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 b/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 index 3ab01da90..a82f710d2 100644 --- a/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/adt_rads_all.yaml.j2 @@ -12,6 +12,11 @@ simulated variables: [absoluteDynamicTopography] io pool: max pool size: 1 +{% if marine_letkf_app | default(false) %} + distribution: + name: Halo + halo size: {{ marine_letkf_dist_halo_size }} +{% endif %} get values: time interpolation: linear obs operator: @@ -62,7 +67,7 @@ name: MetaData/longitude minvalue: 32 maxvalue: 56 -{% if letkf_app | default(false) %} +{% if marine_letkf_app | default(false) %} obs localizations: - localization method: Rossby base value: 100.0e3 diff --git a/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 index c8d428940..d96cff06e 100644 --- a/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 @@ -12,6 +12,11 @@ simulated variables: [seaIceFraction] io pool: max pool size: 1 +{% if marine_letkf_app | default(false) %} + distribution: + name: Halo + halo size: {{ marine_letkf_dist_halo_size }} +{% endif %} get values: time interpolation: linear obs operator: @@ -41,15 +46,15 @@ action: name: inflate error inflation factor: 2.0 -{% if letkf_app | default(false) %} - - filter: Gaussian Thinning - horizontal_mesh: 25 #km - use_reduced_horizontal_grid: true - select_median: true - action: - name: reduce obs space -{% endif %} -{% if letkf_app | default(false) %} +#{% if marine_letkf_app | default(false) %} +# - filter: Gaussian Thinning +# horizontal_mesh: 25 #km +# use_reduced_horizontal_grid: true +# select_median: true +# action: +# name: reduce obs space +#{% endif %} +{% if marine_letkf_app | default(false) %} obs localizations: - localization method: Rossby base value: 100.0e3 diff --git a/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 index c8d428940..d96cff06e 100644 --- a/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 @@ -12,6 +12,11 @@ simulated variables: [seaIceFraction] io pool: max pool size: 1 +{% if marine_letkf_app | default(false) %} + distribution: + name: Halo + halo size: {{ marine_letkf_dist_halo_size }} +{% endif %} get values: time interpolation: linear obs operator: @@ -41,15 +46,15 @@ action: name: inflate error inflation factor: 2.0 -{% if letkf_app | default(false) %} - - filter: Gaussian Thinning - horizontal_mesh: 25 #km - use_reduced_horizontal_grid: true - select_median: true - action: - name: reduce obs space -{% endif %} -{% if letkf_app | default(false) %} +#{% if marine_letkf_app | default(false) %} +# - filter: Gaussian Thinning +# horizontal_mesh: 25 #km +# use_reduced_horizontal_grid: true +# select_median: true +# action: +# name: reduce obs space +#{% endif %} +{% if marine_letkf_app | default(false) %} obs localizations: - localization method: Rossby base value: 100.0e3 diff --git a/parm/jcb-gdas/observations/marine/icec_generic_passive.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_generic_passive.yaml.j2 index 0cb551814..8526710ae 100644 --- a/parm/jcb-gdas/observations/marine/icec_generic_passive.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_generic_passive.yaml.j2 @@ -12,6 +12,11 @@ simulated variables: [seaIceFraction] io pool: max pool size: 1 +{% if marine_letkf_app | default(false) %} + distribution: + name: Halo + halo size: {{ marine_letkf_dist_halo_size }} +{% endif %} get values: time interpolation: linear obs operator: @@ -57,7 +62,7 @@ where: - variable: {name: GeoVaLs/sea_surface_temperature} maxvalue: -4.0 -{% if letkf_app | default(false) %} +{% if marine_letkf_app | default(false) %} obs localizations: - localization method: Rossby base value: 100.0e3 diff --git a/parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 index 700fdfb6c..ac83a5e80 100644 --- a/parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_ssmis_f17_l2.yaml.j2 @@ -12,6 +12,11 @@ simulated variables: [seaIceFraction] io pool: max pool size: 1 +{% if marine_letkf_app | default(false) %} + distribution: + name: Halo + halo size: {{ marine_letkf_dist_halo_size }} +{% endif %} get values: time interpolation: linear obs operator: @@ -57,7 +62,7 @@ # where: # - variable: {name: GeoVaLs/sea_surface_temperature} # maxvalue: -4.0 -{% if letkf_app | default(false) %} +{% if marine_letkf_app | default(false) %} obs localizations: - localization method: Rossby base value: 100.0e3 diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 index ad936b25e..fe8bea759 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_bathy.yaml.j2 @@ -13,6 +13,11 @@ observed variables: [waterTemperature] io pool: max pool size: 1 +{% if marine_letkf_app | default(false) %} + distribution: + name: Halo + halo size: {{ marine_letkf_dist_halo_size }} +{% endif %} obs operator: name: InsituTemperature obs error: @@ -28,7 +33,7 @@ - ObsError/waterTemperature coefs: - 1000.0 -{% if letkf_app | default(false) %} +{% if marine_letkf_app | default(false) %} obs localizations: - localization method: Rossby base value: 100.0e3 diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_glider.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_glider.yaml.j2 index 124799890..513af6ccf 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_glider.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_glider.yaml.j2 @@ -13,6 +13,11 @@ observed variables: [waterTemperature] io pool: max pool size: 1 +{% if marine_letkf_app | default(false) %} + distribution: + name: Halo + halo size: {{ marine_letkf_dist_halo_size }} +{% endif %} obs operator: name: Composite components: @@ -32,7 +37,7 @@ - ObsError/waterTemperature coefs: - 1000.0 -{% if letkf_app | default(false) %} +{% if marine_letkf_app | default(false) %} obs localizations: - localization method: Rossby base value: 100.0e3 diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_marinemammal.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_marinemammal.yaml.j2 index 124799890..513af6ccf 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_marinemammal.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_marinemammal.yaml.j2 @@ -13,6 +13,11 @@ observed variables: [waterTemperature] io pool: max pool size: 1 +{% if marine_letkf_app | default(false) %} + distribution: + name: Halo + halo size: {{ marine_letkf_dist_halo_size }} +{% endif %} obs operator: name: Composite components: @@ -32,7 +37,7 @@ - ObsError/waterTemperature coefs: - 1000.0 -{% if letkf_app | default(false) %} +{% if marine_letkf_app | default(false) %} obs localizations: - localization method: Rossby base value: 100.0e3 diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_tesac.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_tesac.yaml.j2 index a004fefe2..9eb69b6b4 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_tesac.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_tesac.yaml.j2 @@ -13,6 +13,11 @@ observed variables: [waterTemperature] io pool: max pool size: 1 +{% if marine_letkf_app | default(false) %} + distribution: + name: Halo + halo size: {{ marine_letkf_dist_halo_size }} +{% endif %} obs operator: name: Composite components: @@ -30,7 +35,7 @@ - ObsError/waterTemperature coefs: - 1000.0 -{% if letkf_app | default(false) %} +{% if marine_letkf_app | default(false) %} obs localizations: - localization method: Rossby base value: 100.0e3 diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_tesac_salinity.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_tesac_salinity.yaml.j2 index a92e2bf49..6a0fe7553 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_tesac_salinity.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_tesac_salinity.yaml.j2 @@ -13,6 +13,11 @@ observed variables: [salinity] io pool: max pool size: 1 +{% if marine_letkf_app | default(false) %} + distribution: + name: Halo + halo size: {{ marine_letkf_dist_halo_size }} +{% endif %} obs operator: name: VertInterp observation alias file: ./obsop_name_map.yaml @@ -32,7 +37,7 @@ - ObsError/salinity coefs: - 1000.0 -{% if letkf_app | default(false) %} +{% if marine_letkf_app | default(false) %} obs localizations: - localization method: Rossby base value: 100.0e3 diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_tropical.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_tropical.yaml.j2 index 053187439..83c5788b2 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_tropical.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_tropical.yaml.j2 @@ -13,6 +13,11 @@ observed variables: [waterTemperature] io pool: max pool size: 1 +{% if marine_letkf_app | default(false) %} + distribution: + name: Halo + halo size: {{ marine_letkf_dist_halo_size }} +{% endif %} obs operator: name: Composite components: @@ -38,7 +43,7 @@ where: - variable: {name: GeoVaLs/sea_surface_temperature} maxvalue: -4.0 -{% if letkf_app | default(false) %} +{% if marine_letkf_app | default(false) %} obs localizations: - localization method: Rossby base value: 100.0e3 diff --git a/parm/jcb-gdas/observations/marine/insitu_profile_xbtctd.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_profile_xbtctd.yaml.j2 index 124799890..513af6ccf 100644 --- a/parm/jcb-gdas/observations/marine/insitu_profile_xbtctd.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_profile_xbtctd.yaml.j2 @@ -13,6 +13,11 @@ observed variables: [waterTemperature] io pool: max pool size: 1 +{% if marine_letkf_app | default(false) %} + distribution: + name: Halo + halo size: {{ marine_letkf_dist_halo_size }} +{% endif %} obs operator: name: Composite components: @@ -32,7 +37,7 @@ - ObsError/waterTemperature coefs: - 1000.0 -{% if letkf_app | default(false) %} +{% if marine_letkf_app | default(false) %} obs localizations: - localization method: Rossby base value: 100.0e3 diff --git a/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 index 91a76c650..c11b62483 100644 --- a/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_salt_profile_argo.yaml.j2 @@ -18,6 +18,11 @@ observed variables: [salinity] io pool: max pool size: 1 +{% if marine_letkf_app | default(false) %} + distribution: + name: Halo + halo size: {{ marine_letkf_dist_halo_size }} +{% endif %} obs operator: name: VertInterp observation alias file: ./obsop_name_map.yaml @@ -186,7 +191,7 @@ - ObsError/salinity coefs: - 100.0 -{% if letkf_app | default(false) %} +{% if marine_letkf_app | default(false) %} obs localizations: - localization method: Rossby base value: 100.0e3 diff --git a/parm/jcb-gdas/observations/marine/insitu_surface_altkob.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_surface_altkob.yaml.j2 index 2192f9958..aadad95f3 100644 --- a/parm/jcb-gdas/observations/marine/insitu_surface_altkob.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_surface_altkob.yaml.j2 @@ -13,6 +13,11 @@ observed variables: [seaSurfaceTemperature] io pool: max pool size: 1 +{% if marine_letkf_app | default(false) %} + distribution: + name: Halo + halo size: {{ marine_letkf_dist_halo_size }} +{% endif %} obs operator: name: Identity observation alias file: obsop_name_map.yaml @@ -29,7 +34,7 @@ - ObsError/seaSurfaceTemperature coefs: - 1000.0 -{% if letkf_app | default(false) %} +{% if marine_letkf_app | default(false) %} obs localizations: - localization method: Rossby base value: 100.0e3 diff --git a/parm/jcb-gdas/observations/marine/insitu_surface_dbuoyb_drifter.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_surface_dbuoyb_drifter.yaml.j2 index c79cd5b69..b99ea1d04 100644 --- a/parm/jcb-gdas/observations/marine/insitu_surface_dbuoyb_drifter.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_surface_dbuoyb_drifter.yaml.j2 @@ -13,6 +13,11 @@ observed variables: [seaSurfaceTemperature] io pool: max pool size: 1 +{% if marine_letkf_app | default(false) %} + distribution: + name: Halo + halo size: {{ marine_letkf_dist_halo_size }} +{% endif %} get values: time interpolation: linear obs operator: @@ -45,7 +50,7 @@ - variable: {name: GeoVaLs/sea_surface_temperature} minvalue: -1.2 -{% if letkf_app | default(false) %} +{% if marine_letkf_app | default(false) %} obs localizations: - localization method: Rossby base value: 100.0e3 diff --git a/parm/jcb-gdas/observations/marine/insitu_surface_trkob.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_surface_trkob.yaml.j2 index 6dea2041e..c4b8e8b51 100644 --- a/parm/jcb-gdas/observations/marine/insitu_surface_trkob.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_surface_trkob.yaml.j2 @@ -13,6 +13,11 @@ observed variables: [seaSurfaceTemperature] io pool: max pool size: 1 +{% if marine_letkf_app | default(false) %} + distribution: + name: Halo + halo size: {{ marine_letkf_dist_halo_size }} +{% endif %} obs operator: name: Identity observation alias file: ./obsop_name_map.yaml @@ -29,7 +34,7 @@ - ObsError/seaSurfaceTemperature coefs: - 1000.0 -{% if letkf_app | default(false) %} +{% if marine_letkf_app | default(false) %} obs localizations: - localization method: Rossby base value: 100.0e3 diff --git a/parm/jcb-gdas/observations/marine/insitu_surface_trkob_salinity.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_surface_trkob_salinity.yaml.j2 index 80ca4e854..97f4c58f2 100644 --- a/parm/jcb-gdas/observations/marine/insitu_surface_trkob_salinity.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_surface_trkob_salinity.yaml.j2 @@ -11,6 +11,11 @@ obsfile: "{{marine_obsdataout_path}}/{{marine_obsdataout_prefix}}{{observation_from_jcb}}{{marine_obsdataout_suffix}}" simulated variables: [seaSurfaceSalinity] observed variables: [seaSurfaceSalinity] +{% if marine_letkf_app | default(false) %} + distribution: + name: Halo + halo size: {{ marine_letkf_dist_halo_size }} +{% endif %} get values: time interpolation: linear obs operator: @@ -29,7 +34,7 @@ - ObsError/seaSurfaceSalinity coefs: - 1000.0 -{% if letkf_app | default(false) %} +{% if marine_letkf_app | default(false) %} obs localizations: - localization method: Rossby base value: 100.0e3 diff --git a/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 index b193a594b..b2c0572e1 100644 --- a/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_temp_profile_argo.yaml.j2 @@ -18,6 +18,11 @@ observed variables: [waterTemperature] io pool: max pool size: 1 +{% if marine_letkf_app | default(false) %} + distribution: + name: Halo + halo size: {{ marine_letkf_dist_halo_size }} +{% endif %} obs operator: name: InsituTemperature obs error: @@ -182,7 +187,7 @@ - ObsError/waterTemperature coefs: - 100.0 -{% if letkf_app | default(false) %} +{% if marine_letkf_app | default(false) %} obs localizations: - localization method: Rossby base value: 100.0e3 diff --git a/parm/jcb-gdas/observations/marine/insitu_temp_surface_drifter.yaml.j2 b/parm/jcb-gdas/observations/marine/insitu_temp_surface_drifter.yaml.j2 index 2b2830fee..648ee2ed8 100644 --- a/parm/jcb-gdas/observations/marine/insitu_temp_surface_drifter.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/insitu_temp_surface_drifter.yaml.j2 @@ -13,6 +13,11 @@ observed variables: [seaSurfaceTemperature] io pool: max pool size: 1 +{% if marine_letkf_app | default(false) %} + distribution: + name: Halo + halo size: {{ marine_letkf_dist_halo_size }} +{% endif %} get values: time interpolation: linear obs operator: @@ -49,7 +54,7 @@ - filter: Background Check absolute threshold: 5.0 -{% if letkf_app | default(false) %} +{% if marine_letkf_app | default(false) %} obs localizations: - localization method: Rossby base value: 100.0e3 diff --git a/parm/jcb-gdas/observations/marine/sss_smap_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/sss_smap_l2.yaml.j2 index aeaa34517..a075b5545 100644 --- a/parm/jcb-gdas/observations/marine/sss_smap_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sss_smap_l2.yaml.j2 @@ -12,6 +12,11 @@ simulated variables: [seaSurfaceSalinity] io pool: max pool size: 1 +{% if marine_letkf_app | default(false) %} + distribution: + name: Halo + halo size: {{ marine_letkf_dist_halo_size }} +{% endif %} get values: time interpolation: linear obs operator: @@ -44,7 +49,7 @@ where: - variable: {name: GeoVaLs/distance_from_coast} minvalue: 100e3 -{% if letkf_app | default(false) %} +{% if marine_letkf_app | default(false) %} obs localizations: - localization method: Rossby base value: 100.0e3 diff --git a/parm/jcb-gdas/observations/marine/sss_smos_l2.yaml.j2 b/parm/jcb-gdas/observations/marine/sss_smos_l2.yaml.j2 index aeaa34517..a075b5545 100644 --- a/parm/jcb-gdas/observations/marine/sss_smos_l2.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sss_smos_l2.yaml.j2 @@ -12,6 +12,11 @@ simulated variables: [seaSurfaceSalinity] io pool: max pool size: 1 +{% if marine_letkf_app | default(false) %} + distribution: + name: Halo + halo size: {{ marine_letkf_dist_halo_size }} +{% endif %} get values: time interpolation: linear obs operator: @@ -44,7 +49,7 @@ where: - variable: {name: GeoVaLs/distance_from_coast} minvalue: 100e3 -{% if letkf_app | default(false) %} +{% if marine_letkf_app | default(false) %} obs localizations: - localization method: Rossby base value: 100.0e3 diff --git a/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 index b9fd51d5e..d887bf220 100644 --- a/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sst_generic.yaml.j2 @@ -12,6 +12,11 @@ simulated variables: [seaSurfaceTemperature] io pool: max pool size: 1 +{% if marine_letkf_app | default(false) %} + distribution: + name: Halo + halo size: {{ marine_letkf_dist_halo_size }} +{% endif %} get values: time interpolation: linear obs operator: @@ -62,7 +67,7 @@ - filter: Background Check absolute threshold: 5.0 -{% if letkf_app | default(false) %} +{% if marine_letkf_app | default(false) %} obs localizations: - localization method: Rossby base value: 100.0e3 diff --git a/parm/jcb-gdas/observations/marine/sst_generic_geo.yaml.j2 b/parm/jcb-gdas/observations/marine/sst_generic_geo.yaml.j2 index 75b148e0d..ffe625a4e 100644 --- a/parm/jcb-gdas/observations/marine/sst_generic_geo.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/sst_generic_geo.yaml.j2 @@ -12,6 +12,11 @@ simulated variables: [seaSurfaceTemperature] io pool: max pool size: 1 +{% if marine_letkf_app | default(false) %} + distribution: + name: Halo + halo size: {{ marine_letkf_dist_halo_size }} +{% endif %} get values: time interpolation: linear obs operator: @@ -68,7 +73,7 @@ - filter: Background Check absolute threshold: 5.0 -{% if letkf_app | default(false) %} +{% if marine_letkf_app | default(false) %} obs localizations: - localization method: Rossby base value: 100.0e3 From fe5311bd8068648436b12889456feb3660528d93 Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Thu, 23 Oct 2025 09:16:15 -0400 Subject: [PATCH 168/199] Bring back conventional_ps (#195) --- .../atmos/conventional_ps_template.yaml.j2 | 49 +++ .../atmosphere/conventional_ps.yaml.j2 | 368 ++++++++++++++++++ 2 files changed, 417 insertions(+) create mode 100644 parm/jcb-gdas/algorithm/obstats/atmos/conventional_ps_template.yaml.j2 create mode 100644 parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/obstats/atmos/conventional_ps_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/atmos/conventional_ps_template.yaml.j2 new file mode 100644 index 000000000..e38129ab3 --- /dev/null +++ b/parm/jcb-gdas/algorithm/obstats/atmos/conventional_ps_template.yaml.j2 @@ -0,0 +1,49 @@ +- obs space: + name: prepbufr_adpsfc + obsdatain: + engine: + type: H5File + obsfile: {{ atmosphere_obsdatain_path }}/atmos/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc + simulated variables: ['stationPressure'] + observed variables: ['stationPressure'] + statistics to compute: ['mean', 'count', 'RMS'] + variables: ['stationPressure'] + groups to process: ['ombg', 'oman'] + qc groups: ['EffectiveQC0', 'EffectiveQC1'] + output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_atmos.nc" + regular grid binning: + bin size in degrees: 1.0 + use negative longitudes: true + domains to process: + - domain: + name: "SH" + first mask variable: latitude + first mask range: [-90,0] + - domain: + name: "NH" + first mask variable: latitude + first mask range: [0,90] + - domain: + name: "CONUS" + first mask variable: latitude + first mask range: [25,49] + second mask variable: longitude + second mask range: [-125,-66] + - domain: + name: "Europe" + first mask variable: latitude + first mask range: [35,70] + second mask variable: longitude + second mask range: [-11,38] + - domain: + name: "Africa" + first mask variable: latitude + first mask range: [-35,37] + second mask variable: longitude + second mask range: [-17,52] + - domain: + name: "Asia" + first mask variable: latitude + first mask range: [0,70] + second mask variable: longitude + second mask range: [38, 180] \ No newline at end of file diff --git a/parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 new file mode 100644 index 000000000..c50b528e9 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 @@ -0,0 +1,368 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: surface_ps + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [stationPressure] + + # Observation Operator + # -------------------- + obs operator: + name: SfcCorrected + variables: + - name: stationPressure + correction scheme to use: GSL + station_altitude: height + geovar_sfc_geomz: height_above_mean_sea_level_at_surface + geovar_geomz: geopotential_height + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: Identity + variables: + - name: stationPressure + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Initial Error Assignments for SFC Observations + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [181] + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + round_to_the_nearest_integer: true + xvar: + name: ObsValue/stationPressure + xvals: [80000, 75000, 70000, 65000, 60000, 55000] + errors: [110, 120, 120, 120, 120, 1.0e+11] + + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [187] + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + round_to_the_nearest_integer: true + xvar: + name: ObsValue/stationPressure + xvals: [85000, 80000, 75000, 70000, 65000, 60000, 55000] + errors: [120, 140, 140, 140, 140, 140, 1.0e+11] + + # Initial Error Assignments for SFCSHIP Observations + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [180] + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + round_to_the_nearest_integer: true + xvar: + name: ObsValue/stationPressure + xvals: [60000, 55000] + errors: [130, 1.0e+11] + + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [183] + action: + name: assign error + error parameter: 1.0e+11 + + # Initial Error Assignments for Radiosonde + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: ObsType/stationPressure + is_in: [120] + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + round_to_the_nearest_integer: true + xvar: + name: ObsValue/stationPressure + xvals: [80000, 75000, 70000, 65000, 60000, 55000] + errors: [110, 120, 120, 120, 120, 1.0e+11] + + # Create PreQC group variable (pqm in GSI read_prepbufr) + - filter: Variable Assignment + assignments: + - name: InputObsError/stationPressure + type: float + source variable: ObsErrorData/stationPressure + + # Set observation quality-realted variables + # Create PreQC group variable (pqm in GSI read_prepbufr) + - filter: Variable Assignment + assignments: + - name: PreQC/stationPressure + type: int + source variable: QualityMarker/stationPressure + + # Create PreUseFlag group variable (usage in GSI read_prepbufr) + # Initialize + - filter: Variable Assignment + assignments: + - name: PreUseFlag/stationPressure + type: int + source variable: PreQC/stationPressure + + - filter: Variable Assignment + where: + - variable: + name: PreUseFlag/stationPressure + is_in: 1-15 + assignments: + - name: PreUseFlag/stationPressure + value: 0 + # Re-assignment + - filter: Variable Assignment + where: + - variable: + name: ObsType/stationPressure + is_in: 183 + assignments: + - name: PreUseFlag/stationPressure + value: 100 + + - filter: Variable Assignment + where: + - variable: + name: ObsValue/stationPressure + is_defined: + - variable: + name: ObsValue/stationPressure + maxvalue: 50000.00 + where operator: and + assignments: + - name: PreUseFlag/stationPressure + value: 100 + + - filter: Variable Assignment + where: + - variable: + name: PreQC/stationPressure + is_in: 9, 12, 15 + assignments: + - name: PreUseFlag/stationPressure + value: 100 + + - filter: Variable Assignment + where: + - variable: + name: PreQC/stationPressure + is_in: 4-15 + assignments: + - name: PreUseFlag/stationPressure + value: 101 + + ## Observation range sanity check + #- filter: Bounds Check + # filter variables: + # - name: stationPressure + # minvalue: 37499.0 + # maxvalue: 106999.0 + # action: + # name: reject + ## Reject all ObsType 183 + #- filter: RejectList + # where: + # - variable: + # name: ObsType/stationPressure + # is_in: 183 + ## Reject surface pressure below 500 hPa + #- filter: Bounds Check + # filter variables: + # - name: stationPressure + # minvalue: 50000.00 + # action: + # name: reject + #- filter: RejectList + # where: + # - variable: + # name: PreQC/stationPressure + # is_in: 4-15 + # Inflate obs error based on obs type + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: PreQC/stationPressure + is_in: 3, 7 + action: + name: inflate error + inflation factor: 1.2 + + # Observation Post Filters (QC) + # ----------------------------- + obs post filters: + # Calculate obs error inflation factors for duplicated observations at the same location + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorDuplicateCheck/stationPressure + type: float + function: + name: ObsFunction/ObsErrorFactorDuplicateCheck + options: + use_air_pressure: false + variable: stationPressure + + # Reduce effective observation error based on obs type and subtype + # In this case: reduce effective obs error for buoy + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: + name: ObsType/stationPressure + is_in: 180 + - variable: + name: ObsSubType/stationPressure + is_in: 0 + action: + name: inflate error + inflation factor: 0.7 + + # Calculate obs error inflation factors for large discrepancies between model and observations + - filter: Variable Assignment + assignments: + - name: ObsErrorFactorSfcPressure/stationPressure + type: float + function: + name: ObsFunction/ObsErrorFactorSfcPressure + options: + geovar_sfc_geomz: height_above_mean_sea_level_at_surface + geovar_geomz: geopotential_height + station_altitude: height + + # Inflate surface pressure observation based on discrepancies between + # model and observations due to terrian + - filter: Perform Action + filter variables: + - name: stationPressure + action: + name: inflate error + inflation variable: + name: ObsErrorFactorSfcPressure/stationPressure + + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/Innovation + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: ObsValue/stationPressure + - name: HofX/stationPressure + coefs: [1, -1] + + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ObsErrorBoundSfcPressure1 + type: float + function: + name: ObsFunction/ObsErrorBoundConventional + options: + obsvar: stationPressure + obserr_bound_min: 100 + obserr_bound_max: 300 + obserr_bound_factor: 5.0 + + - filter: Background Check + filter variables: + - name: stationPressure + where: + - variable: PreQC/stationPressure + is_not_in: 3 + function absolute threshold: + - name: DerivedMetaData/ObsErrorBoundSfcPressure1 + action: + name: reject + + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ObsErrorBoundSfcPressure2 + type: float + function: + name: ObsFunction/ObsErrorBoundConventional + options: + obsvar: stationPressure + obserr_bound_min: 100 + obserr_bound_max: 300 + obserr_bound_factor: 3.5 + + - filter: Background Check + filter variables: + - name: stationPressure + where: + - variable: PreQC/stationPressure + is_in: 3 + function absolute threshold: + - name: DerivedMetaData/ObsErrorBoundSfcPressure2 + action: + name: reject + + # Inflate obs error based on duplicate check + - filter: Perform Action + filter variables: + - name: stationPressure + action: + name: inflate error + inflation variable: + name: ObsErrorFactorDuplicateCheck/stationPressure + + # Reject data based on PreUseFlag (usage in GSI) + - filter: Perform Action + filter variables: + - name: stationPressure + where: + - variable: PreUseFlag/stationPressure + is_not_in: 0, 1 + action: + name: reject + # End of Filters + + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 \ No newline at end of file From 8d72456ac053af8cc1b24f4966211fce06a95f41 Mon Sep 17 00:00:00 2001 From: Jiarui Dong Date: Fri, 24 Oct 2025 08:04:22 -0400 Subject: [PATCH 169/199] Modify the maximum value of station elevation to 3000 m (#196) This PR implements the exclusion of observations with elevations above 3000 meters in the 2DVar snow DA. --- parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 | 2 +- parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 | 2 +- parm/jcb-gdas/observations/snow/ims_snow.yaml.j2 | 13 +++++++++++++ parm/jcb-gdas/observations/snow/madis_snow.yaml.j2 | 2 +- parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 | 2 +- parm/jcb-gdas/observations/snow/snocvr.yaml.j2 | 2 +- .../observations/snow/snocvr_snomad.yaml.j2 | 2 +- parm/jcb-gdas/observations/snow/snomad.yaml.j2 | 2 +- 8 files changed, 20 insertions(+), 7 deletions(-) diff --git a/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 index 596dd3ffd..834e3b37a 100644 --- a/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 @@ -114,7 +114,7 @@ - variable: name: MetaData/stationElevation minvalue: -200.0 - maxvalue: 9900.0 + maxvalue: 3000.0 actions: - name: set flag: invalid_elevation diff --git a/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 index f8fd5f839..668c160bb 100644 --- a/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 @@ -99,7 +99,7 @@ - variable: name: MetaData/stationElevation minvalue: -200.0 - maxvalue: 9900.0 + maxvalue: 3000.0 actions: - name: set flag: invalid_elevation diff --git a/parm/jcb-gdas/observations/snow/ims_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/ims_snow.yaml.j2 index a40980503..1f1b8e77c 100644 --- a/parm/jcb-gdas/observations/snow/ims_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/ims_snow.yaml.j2 @@ -35,6 +35,8 @@ initial value: false - name: invalid_snowdepth initial value: false + - name: invalid_elevation + initial value: false - name: background_check initial value: false - name: spatial_thinning @@ -59,6 +61,17 @@ flag: land_check ignore: rejected observations - name: reject + - filter: Domain Check + where: + - variable: + name: MetaData/stationElevation + minvalue: -200.0 + maxvalue: 3000.0 + actions: + - name: set + flag: invalid_elevation + ignore: rejected observations + - name: reject - filter: Domain Check # land only, no sea ice where: - variable: diff --git a/parm/jcb-gdas/observations/snow/madis_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/madis_snow.yaml.j2 index c388ffd91..23339c78c 100644 --- a/parm/jcb-gdas/observations/snow/madis_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/madis_snow.yaml.j2 @@ -113,7 +113,7 @@ - variable: name: MetaData/stationElevation minvalue: -200.0 - maxvalue: 9900.0 + maxvalue: 3000.0 actions: - name: set flag: invalid_elevation diff --git a/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 b/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 index df423d826..d1683eca1 100644 --- a/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 @@ -114,7 +114,7 @@ - variable: name: MetaData/stationElevation minvalue: -200.0 - maxvalue: 9900.0 + maxvalue: 3000.0 actions: - name: set flag: invalid_elevation diff --git a/parm/jcb-gdas/observations/snow/snocvr.yaml.j2 b/parm/jcb-gdas/observations/snow/snocvr.yaml.j2 index f9b38a102..7785cf1a1 100644 --- a/parm/jcb-gdas/observations/snow/snocvr.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/snocvr.yaml.j2 @@ -114,7 +114,7 @@ - variable: name: MetaData/stationElevation minvalue: -200.0 - maxvalue: 9900.0 + maxvalue: 3000.0 actions: - name: set flag: invalid_elevation diff --git a/parm/jcb-gdas/observations/snow/snocvr_snomad.yaml.j2 b/parm/jcb-gdas/observations/snow/snocvr_snomad.yaml.j2 index ade714cfd..865d9ae2d 100644 --- a/parm/jcb-gdas/observations/snow/snocvr_snomad.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/snocvr_snomad.yaml.j2 @@ -113,7 +113,7 @@ - variable: name: MetaData/stationElevation minvalue: -200.0 - maxvalue: 9900.0 + maxvalue: 3000.0 actions: - name: set flag: invalid_elevation diff --git a/parm/jcb-gdas/observations/snow/snomad.yaml.j2 b/parm/jcb-gdas/observations/snow/snomad.yaml.j2 index 678d9cf99..cccb13bdd 100644 --- a/parm/jcb-gdas/observations/snow/snomad.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/snomad.yaml.j2 @@ -114,7 +114,7 @@ - variable: name: MetaData/stationElevation minvalue: -200.0 - maxvalue: 9900.0 + maxvalue: 3000.0 actions: - name: set flag: invalid_elevation From 3d8899d3c60f5e564aebc039b642ed089c39446e Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Tue, 28 Oct 2025 08:27:42 -0400 Subject: [PATCH 170/199] Add missing file action warn to templates (#197) Added `missing file action: warn` to all atmospheric ob templates --- parm/jcb-gdas/observations/atmosphere/abi_g16.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/abi_g17.yaml.j2 | 1 + .../jcb-gdas/observations/atmosphere/aircraft_humidity.yaml.j2 | 1 + .../observations/atmosphere/aircraft_temperature.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/aircraft_wind.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/amsr2_gcom-w1.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/cris-fsr_n21.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/cris-fsr_npp.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/gnssro.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/gnssro_cosmic2.yaml.j2 | 3 ++- parm/jcb-gdas/observations/atmosphere/gnssro_geoopt.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/gnssro_grace.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/gnssro_k5.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/gnssro_metop.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/gnssro_paz.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/gnssro_piq.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/gnssro_s6.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/gnssro_spire.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/gnssro_tdm.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/gnssro_tsx.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/iasi_metop-c.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/ozone.omi_aura.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_n20.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_npp.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/ozone.ompstc_n20.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/ozone.ompstc_npp.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/prepbufr_adpsfc.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/prepbufr_adpupa.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/prepbufr_sfcshp.yaml.j2 | 1 + .../observations/atmosphere/satwnd.abi_goes-16.yaml.j2 | 1 + .../observations/atmosphere/satwnd.abi_goes-17.yaml.j2 | 1 + .../observations/atmosphere/satwnd.abi_goes-18.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h8.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h9.yaml.j2 | 1 + .../observations/atmosphere/satwnd.avhrr_metop-a.yaml.j2 | 1 + .../observations/atmosphere/satwnd.avhrr_metop-b.yaml.j2 | 1 + .../observations/atmosphere/satwnd.avhrr_metop-c.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n15.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n18.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n19.yaml.j2 | 1 + .../observations/atmosphere/satwnd.leogeo_multi.yaml.j2 | 1 + .../jcb-gdas/observations/atmosphere/satwnd.modis_aqua.yaml.j2 | 1 + .../observations/atmosphere/satwnd.modis_terra.yaml.j2 | 1 + .../jcb-gdas/observations/atmosphere/satwnd.seviri_m10.yaml.j2 | 1 + .../jcb-gdas/observations/atmosphere/satwnd.seviri_m11.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m8.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m9.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/satwnd.viirs_n20.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/satwnd.viirs_npp.yaml.j2 | 1 + .../observations/atmosphere/scatwnd.ascat_metop-a.yaml.j2 | 1 + .../observations/atmosphere/scatwnd.ascat_metop-b.yaml.j2 | 1 + .../observations/atmosphere/scatwnd.ascat_metop-c.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/seviri_m08.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/seviri_m11.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/sfc.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/sfcship.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 | 1 + parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 | 1 + 67 files changed, 68 insertions(+), 1 deletion(-) diff --git a/parm/jcb-gdas/observations/atmosphere/abi_g16.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/abi_g16.yaml.j2 index 9d0d0d673..441d53889 100644 --- a/parm/jcb-gdas/observations/atmosphere/abi_g16.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/abi_g16.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/abi_g17.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/abi_g17.yaml.j2 index dc8145bae..beae2197d 100644 --- a/parm/jcb-gdas/observations/atmosphere/abi_g17.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/abi_g17.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/aircraft_humidity.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/aircraft_humidity.yaml.j2 index 8bc9cd746..1f30ddb31 100644 --- a/parm/jcb-gdas/observations/atmosphere/aircraft_humidity.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/aircraft_humidity.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsgrouping: group variables: [stationIdentification] sort variable: pressure diff --git a/parm/jcb-gdas/observations/atmosphere/aircraft_temperature.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/aircraft_temperature.yaml.j2 index 1c50f9064..09129d45b 100644 --- a/parm/jcb-gdas/observations/atmosphere/aircraft_temperature.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/aircraft_temperature.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsgrouping: group variables: [stationIdentification] sort variable: pressure diff --git a/parm/jcb-gdas/observations/atmosphere/aircraft_wind.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/aircraft_wind.yaml.j2 index a2ef0e54b..0063393ff 100644 --- a/parm/jcb-gdas/observations/atmosphere/aircraft_wind.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/aircraft_wind.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsgrouping: group variables: [stationIdentification] sort variable: pressure diff --git a/parm/jcb-gdas/observations/atmosphere/amsr2_gcom-w1.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/amsr2_gcom-w1.yaml.j2 index 31344ebff..32759dbe4 100644 --- a/parm/jcb-gdas/observations/atmosphere/amsr2_gcom-w1.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/amsr2_gcom-w1.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 index d10346d8f..fe0698b5d 100644 --- a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 index 82e048949..5812f0b04 100644 --- a/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 index 63da3fa52..1a39ca6e6 100644 --- a/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 index cdb0e4ef2..e55ee691f 100644 --- a/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 index c50b528e9..5f7ffaff8 100644 --- a/parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 index a1640e4ad..b59b83db9 100644 --- a/parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/cris-fsr_n21.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/cris-fsr_n21.yaml.j2 index d9737d191..aa91769a7 100644 --- a/parm/jcb-gdas/observations/atmosphere/cris-fsr_n21.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/cris-fsr_n21.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/cris-fsr_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/cris-fsr_npp.yaml.j2 index 47603300c..8a0e8fb23 100644 --- a/parm/jcb-gdas/observations/atmosphere/cris-fsr_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/cris-fsr_npp.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro.yaml.j2 index 6aab75efa..a57f604ae 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn # obsgrouping: # group variables: [ "sequenceNumber" ] # sort variable: "impactHeightRO" diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_cosmic2.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_cosmic2.yaml.j2 index fa3f48869..e18033dd4 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_cosmic2.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_cosmic2.yaml.j2 @@ -7,7 +7,8 @@ obsdatain: engine: type: H5File - obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsgrouping: group variables: [ 'sequenceNumber' ] sort variable: 'impactHeightRO' diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_geoopt.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_geoopt.yaml.j2 index a7774c545..1ac81847d 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_geoopt.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_geoopt.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsgrouping: group variables: [ 'sequenceNumber' ] sort variable: 'impactHeightRO' diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_grace.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_grace.yaml.j2 index 5a459de8d..8f4f4ad7a 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_grace.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_grace.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsgrouping: group variables: [ 'sequenceNumber' ] sort variable: 'impactHeightRO' diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_k5.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_k5.yaml.j2 index 72893f481..6a7e60c23 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_k5.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_k5.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsgrouping: group variables: [ 'sequenceNumber' ] sort variable: 'impactHeightRO' diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_metop.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_metop.yaml.j2 index 199424b53..be786d247 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_metop.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_metop.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsgrouping: group variables: [ 'sequenceNumber' ] sort variable: 'impactHeightRO' diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_paz.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_paz.yaml.j2 index c36e8923a..861115fc2 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_paz.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_paz.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsgrouping: group variables: [ 'sequenceNumber' ] sort variable: 'impactHeightRO' diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_piq.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_piq.yaml.j2 index ad4686b9f..03f89dd43 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_piq.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_piq.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsgrouping: group variables: [ 'sequenceNumber' ] sort variable: 'impactHeightRO' diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_s6.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_s6.yaml.j2 index bbb0698b1..614f4a1ac 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_s6.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_s6.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsgrouping: group variables: [ 'sequenceNumber' ] sort variable: 'impactHeightRO' diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_spire.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_spire.yaml.j2 index 568da6de7..6cff8d0fe 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_spire.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_spire.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsgrouping: group variables: [ 'sequenceNumber' ] sort variable: 'impactHeightRO' diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_tdm.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_tdm.yaml.j2 index fb38f0126..448fd2889 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_tdm.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_tdm.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsgrouping: group variables: [ 'sequenceNumber' ] sort variable: 'impactHeightRO' diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_tsx.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_tsx.yaml.j2 index 107308486..2539e6888 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_tsx.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_tsx.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsgrouping: group variables: [ 'sequenceNumber' ] sort variable: 'impactHeightRO' diff --git a/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 index 15e1f0ff0..155d5acd2 100644 --- a/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 index 8d869ca5b..1133d2bb2 100644 --- a/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/iasi_metop-c.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/iasi_metop-c.yaml.j2 index ced043808..965e2b55d 100644 --- a/parm/jcb-gdas/observations/atmosphere/iasi_metop-c.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/iasi_metop-c.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/ozone.omi_aura.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ozone.omi_aura.yaml.j2 index d04c894d5..99a31c703 100644 --- a/parm/jcb-gdas/observations/atmosphere/ozone.omi_aura.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ozone.omi_aura.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_n20.yaml.j2 index 6c3033fef..98e855e5c 100644 --- a/parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_n20.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsgrouping: group variables: ["latitude"] sort variable: "pressure" diff --git a/parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_npp.yaml.j2 index 161ab9365..b1e70e6cd 100644 --- a/parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_npp.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsgrouping: group variables: ["latitude"] sort variable: "pressure" diff --git a/parm/jcb-gdas/observations/atmosphere/ozone.ompstc_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ozone.ompstc_n20.yaml.j2 index 2d369a30e..6639fa7df 100644 --- a/parm/jcb-gdas/observations/atmosphere/ozone.ompstc_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ozone.ompstc_n20.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/ozone.ompstc_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ozone.ompstc_npp.yaml.j2 index 47596cfe4..e20cf11ed 100644 --- a/parm/jcb-gdas/observations/atmosphere/ozone.ompstc_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ozone.ompstc_npp.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/prepbufr_adpsfc.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/prepbufr_adpsfc.yaml.j2 index ad79038c0..584c548a0 100644 --- a/parm/jcb-gdas/observations/atmosphere/prepbufr_adpsfc.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/prepbufr_adpsfc.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/prepbufr_adpupa.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/prepbufr_adpupa.yaml.j2 index ad79038c0..584c548a0 100644 --- a/parm/jcb-gdas/observations/atmosphere/prepbufr_adpupa.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/prepbufr_adpupa.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/prepbufr_sfcshp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/prepbufr_sfcshp.yaml.j2 index 39ddc387d..d00e23eb9 100644 --- a/parm/jcb-gdas/observations/atmosphere/prepbufr_sfcshp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/prepbufr_sfcshp.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 index 163b29c22..1ba275681 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-17.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-17.yaml.j2 index 86a799712..a24e094b0 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-17.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-17.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-18.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-18.yaml.j2 index b1a7a6ed7..e775b87ee 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-18.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-18.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h8.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h8.yaml.j2 index 424858e68..c39792cbc 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h8.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h8.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h9.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h9.yaml.j2 index 9d8c6f0c3..467014e79 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h9.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h9.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-a.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-a.yaml.j2 index 45ce9cd58..1528a38d3 100755 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-a.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-a.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-b.yaml.j2 index 20d12c051..5c18ce8ff 100755 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-b.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-b.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-c.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-c.yaml.j2 index 314935408..2edcf7b3f 100755 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-c.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-c.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n15.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n15.yaml.j2 index 93483bb26..373200745 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n15.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n15.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n18.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n18.yaml.j2 index 8fcd566d4..c29900be6 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n18.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n18.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n19.yaml.j2 index 357bc91e2..3b549d5c6 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n19.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n19.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.leogeo_multi.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.leogeo_multi.yaml.j2 index 11ca76e10..3a396b7b7 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.leogeo_multi.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.leogeo_multi.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.modis_aqua.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.modis_aqua.yaml.j2 index c539902bd..f4ebcb18b 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.modis_aqua.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.modis_aqua.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.modis_terra.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.modis_terra.yaml.j2 index 808a85a59..f8722f4b2 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.modis_terra.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.modis_terra.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m10.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m10.yaml.j2 index 0e0090220..94b5207a7 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m10.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m10.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m11.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m11.yaml.j2 index c6523c1c8..e105a14c1 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m11.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m11.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m8.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m8.yaml.j2 index b7cde2045..d32c4dad4 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m8.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m8.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m9.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m9.yaml.j2 index d5f4fac8a..d0ff40087 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m9.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m9.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_n20.yaml.j2 index 32b1afe34..5984fe02b 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_n20.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_npp.yaml.j2 index 2d32a2282..96cc59ebf 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_npp.yaml.j2 @@ -4,6 +4,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-a.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-a.yaml.j2 index 2faa053a8..3af852f3f 100644 --- a/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-a.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-a.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-b.yaml.j2 index 713b9d3b1..286631f6a 100644 --- a/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-b.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-b.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-c.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-c.yaml.j2 index ec1c5e39e..86f00cd9f 100644 --- a/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-c.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-c.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/seviri_m08.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/seviri_m08.yaml.j2 index 29936f758..ac826f86f 100644 --- a/parm/jcb-gdas/observations/atmosphere/seviri_m08.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/seviri_m08.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/seviri_m11.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/seviri_m11.yaml.j2 index 3b49a02e6..4fe5bad46 100644 --- a/parm/jcb-gdas/observations/atmosphere/seviri_m11.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/seviri_m11.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/sfc.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/sfc.yaml.j2 index f659d7dda..0848cc06e 100644 --- a/parm/jcb-gdas/observations/atmosphere/sfc.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/sfc.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/sfcship.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/sfcship.yaml.j2 index 6d0c4980a..9da9a6124 100644 --- a/parm/jcb-gdas/observations/atmosphere/sfcship.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/sfcship.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 index e965d865d..7ffccbb73 100644 --- a/parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsgrouping: group variables: ["stationIdentification"] sort variable: "pressure" diff --git a/parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 index 6081fa373..f81b13d07 100644 --- a/parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 index f42e07e0b..b5578d55b 100644 --- a/parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 @@ -8,6 +8,7 @@ engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn obsdataout: engine: type: H5File From 8e2592ff8e7b7e45399dc4731e95634d0b1dd897 Mon Sep 17 00:00:00 2001 From: Anna Shlyaeva Date: Tue, 28 Oct 2025 09:59:05 -0600 Subject: [PATCH 171/199] Bring back thinning for ice obs with LETKF (#198) Brings back thinning for ice obs with LETKF. This option requires `use linear observer: true` in the LETKF, because the observations are thinned with option `reduce obs space`. When obs space is reduced, it's important to only do it once. `use linear observer: true` ensures that the QC filters are only run once, on the ensemble mean. When `use linear observer` is set to `false` (by default), QC filters are run on each ensemble member, which results in removing observations from the obs space multiple times and subsequent segfaults in the LETKF when the various vector sizes don't match. Goes with https://github.com/NOAA-EMC/jcb-algorithms/pull/16 and a GDASApp PR. Co-authored-by: shlyaeva Co-authored-by: DavidNew-NOAA <134300700+DavidNew-NOAA@users.noreply.github.com> --- .../observations/marine/icec_amsr2_north.yaml.j2 | 16 ++++++++-------- .../observations/marine/icec_amsr2_south.yaml.j2 | 16 ++++++++-------- 2 files changed, 16 insertions(+), 16 deletions(-) diff --git a/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 index d96cff06e..5696bdc66 100644 --- a/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_amsr2_north.yaml.j2 @@ -46,14 +46,14 @@ action: name: inflate error inflation factor: 2.0 -#{% if marine_letkf_app | default(false) %} -# - filter: Gaussian Thinning -# horizontal_mesh: 25 #km -# use_reduced_horizontal_grid: true -# select_median: true -# action: -# name: reduce obs space -#{% endif %} +{% if marine_letkf_app | default(false) %} + - filter: Gaussian Thinning + horizontal_mesh: 25 #km + use_reduced_horizontal_grid: true + select_median: true + action: + name: reduce obs space +{% endif %} {% if marine_letkf_app | default(false) %} obs localizations: - localization method: Rossby diff --git a/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 b/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 index d96cff06e..5696bdc66 100644 --- a/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 +++ b/parm/jcb-gdas/observations/marine/icec_amsr2_south.yaml.j2 @@ -46,14 +46,14 @@ action: name: inflate error inflation factor: 2.0 -#{% if marine_letkf_app | default(false) %} -# - filter: Gaussian Thinning -# horizontal_mesh: 25 #km -# use_reduced_horizontal_grid: true -# select_median: true -# action: -# name: reduce obs space -#{% endif %} +{% if marine_letkf_app | default(false) %} + - filter: Gaussian Thinning + horizontal_mesh: 25 #km + use_reduced_horizontal_grid: true + select_median: true + action: + name: reduce obs space +{% endif %} {% if marine_letkf_app | default(false) %} obs localizations: - localization method: Rossby From f352a1ca84106d67a23ecc386e78a30d79ac67fa Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Tue, 28 Oct 2025 18:20:54 -0400 Subject: [PATCH 172/199] Fix background error filenames in marine hybrid diffusion configuration (#200) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The marine hybrid diffusion background error configuration was using incorrect filenames with `bkgerr_ens_stddev` instead of `bkgerr_parametric_stddev`. ## Changes - Updated `ocn_filename` from `ocn.bkgerr_ens_stddev.nc` to `ocn.bkgerr_parametric_stddev.nc` - Updated `ice_filename` from `ice.bkgerr_ens_stddev.nc` to `ice.bkgerr_parametric_stddev.nc` This aligns the naming convention with `marine_background_error_static_diffusion.yaml.j2` which already uses the correct parametric pattern. ```yaml # In model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 saber outer blocks: - saber block name: StdDev read: model file: date: '{{marine_stddev_time}}' basename: ./staticb/ - ocn_filename: 'ocn.bkgerr_ens_stddev.nc' - ice_filename: 'ice.bkgerr_ens_stddev.nc' + ocn_filename: 'ocn.bkgerr_parametric_stddev.nc' + ice_filename: 'ice.bkgerr_parametric_stddev.nc' read_from_file: 3 ```
Original prompt > On marine_background_error_hybrid_diffusion_diffusion.yaml.j2, the name of the background error file is wrong, {ocn,ice}.bkgerr_ens_stddev.nc should be {ocn,ice}.bkgerr_parametric_stddev.nc
--- 💬 We'd love your input! Share your thoughts on Copilot coding agent in our [2 minute survey](https://gh.io/copilot-coding-agent-survey). --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: guillaumevernieres <14031856+guillaumevernieres@users.noreply.github.com> --- ...marine_background_error_hybrid_diffusion_diffusion.yaml.j2 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/parm/jcb-gdas/model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 b/parm/jcb-gdas/model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 index 2833fd422..6ff6713a4 100644 --- a/parm/jcb-gdas/model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 +++ b/parm/jcb-gdas/model/marine/marine_background_error_hybrid_diffusion_diffusion.yaml.j2 @@ -30,8 +30,8 @@ components: model file: date: '{{marine_stddev_time}}' basename: ./staticb/ - ocn_filename: 'ocn.bkgerr_ens_stddev.nc' - ice_filename: 'ice.bkgerr_ens_stddev.nc' + ocn_filename: 'ocn.bkgerr_parametric_stddev.nc' + ice_filename: 'ice.bkgerr_parametric_stddev.nc' read_from_file: 3 linear variable change: input variables: From aa025029f7f1fb0cee9d88b34c4030b06801332a Mon Sep 17 00:00:00 2001 From: Jiarui Dong Date: Fri, 31 Oct 2025 10:50:16 -0400 Subject: [PATCH 173/199] Add snomad and snocvr_snomad ioda-stats yamls (#202) This PR adds the ioda-stats YAMLs for newly observations including snomad and snocvr_snomad. This PR contributes to https://github.com/NOAA-EMC/GDASApp/pull/1966 --- .../snow/snocvr_snomad_template.yaml.j2 | 68 +++++++++++++++++++ .../obstats/snow/snomad_template.yaml.j2 | 68 +++++++++++++++++++ 2 files changed, 136 insertions(+) create mode 100644 parm/jcb-gdas/algorithm/obstats/snow/snocvr_snomad_template.yaml.j2 create mode 100644 parm/jcb-gdas/algorithm/obstats/snow/snomad_template.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/obstats/snow/snocvr_snomad_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/snow/snocvr_snomad_template.yaml.j2 new file mode 100644 index 000000000..7eb522576 --- /dev/null +++ b/parm/jcb-gdas/algorithm/obstats/snow/snocvr_snomad_template.yaml.j2 @@ -0,0 +1,68 @@ +- obs space: + name: snocvr_snomad + obsdatain: + engine: + type: H5File + obsfile: {{ snow_obsdatain_path }}/snow/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc + simulated variables: ['totalSnowDepth'] + observed variables: ['totalSnowDepth'] + statistics to compute: ['mean', 'count', 'RMS'] + variables: [totalSnowDepth] + groups to process: ['ombg', 'oman'] + qc groups: ['EffectiveQC0', 'EffectiveQC1'] + output file: "{{ stat_current_cycle_YMDH }}.ioda_hofx_stats.{{ obspace }}.nc" + regular grid binning: + bin size in degrees: 1.0 + use negative longitudes: true + domains to process: + - domain: + name: "SH" + first mask variable: latitude + first mask range: [-90,0] + - domain: + name: "NH" + first mask variable: latitude + first mask range: [0,90] + - domain: + name: "West_CONUS" + first mask variable: latitude + first mask range: [25,49] + second mask variable: longitude + second mask range: [-125,-100] + - domain: + name: "East_CONUS" + first mask variable: latitude + first mask range: [25,49] + second mask variable: longitude + second mask range: [-100,-66] + - domain: + name: "Alaska" + first mask variable: latitude + first mask range: [52,72] + second mask variable: longitude + second mask range: [-170,-130] + - domain: + name: "Europe" + first mask variable: latitude + first mask range: [35,70] + second mask variable: longitude + second mask range: [-11,38] + - domain: + name: "Africa" + first mask variable: latitude + first mask range: [-35,37] + second mask variable: longitude + second mask range: [-17,52] + - domain: + name: "Asia" + first mask variable: latitude + first mask range: [0,70] + second mask variable: longitude + second mask range: [38, 180] + - domain: + name: "High_Mount_Asia" + first mask variable: latitude + first mask range: [20,46] + second mask variable: longitude + second mask range: [60, 111] + diff --git a/parm/jcb-gdas/algorithm/obstats/snow/snomad_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/snow/snomad_template.yaml.j2 new file mode 100644 index 000000000..3ecd99856 --- /dev/null +++ b/parm/jcb-gdas/algorithm/obstats/snow/snomad_template.yaml.j2 @@ -0,0 +1,68 @@ +- obs space: + name: snomad + obsdatain: + engine: + type: H5File + obsfile: {{ snow_obsdatain_path }}/snow/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc + simulated variables: ['totalSnowDepth'] + observed variables: ['totalSnowDepth'] + statistics to compute: ['mean', 'count', 'RMS'] + variables: [totalSnowDepth] + groups to process: ['ombg', 'oman'] + qc groups: ['EffectiveQC0', 'EffectiveQC1'] + output file: "{{ stat_current_cycle_YMDH }}.ioda_hofx_stats.{{ obspace }}.nc" + regular grid binning: + bin size in degrees: 1.0 + use negative longitudes: true + domains to process: + - domain: + name: "SH" + first mask variable: latitude + first mask range: [-90,0] + - domain: + name: "NH" + first mask variable: latitude + first mask range: [0,90] + - domain: + name: "West_CONUS" + first mask variable: latitude + first mask range: [25,49] + second mask variable: longitude + second mask range: [-125,-100] + - domain: + name: "East_CONUS" + first mask variable: latitude + first mask range: [25,49] + second mask variable: longitude + second mask range: [-100,-66] + - domain: + name: "Alaska" + first mask variable: latitude + first mask range: [52,72] + second mask variable: longitude + second mask range: [-170,-130] + - domain: + name: "Europe" + first mask variable: latitude + first mask range: [35,70] + second mask variable: longitude + second mask range: [-11,38] + - domain: + name: "Africa" + first mask variable: latitude + first mask range: [-35,37] + second mask variable: longitude + second mask range: [-17,52] + - domain: + name: "Asia" + first mask variable: latitude + first mask range: [0,70] + second mask variable: longitude + second mask range: [38, 180] + - domain: + name: "High_Mount_Asia" + first mask variable: latitude + first mask range: [20,46] + second mask variable: longitude + second mask range: [60, 111] + From b70698963806580752df3c65869a14786537c51f Mon Sep 17 00:00:00 2001 From: David Huber <69919478+DavidHuber-NOAA@users.noreply.github.com> Date: Fri, 31 Oct 2025 12:50:07 -0400 Subject: [PATCH 174/199] Update snow filenames to comply with EE2 standards (#201) This updates the output filenames for the observation statistics files to comply with EE2 standards. Prerequisite for https://github.com/NOAA-EMC/global-workflow/pull/4195. Currently in testing. --- .../algorithm/obstats/snow/ims_snow_template.yaml.j2 | 4 ++-- .../algorithm/obstats/snow/madis_snow_template.yaml.j2 | 6 +++--- .../jcb-gdas/algorithm/obstats/snow/sfcsno_template.yaml.j2 | 4 ++-- .../jcb-gdas/algorithm/obstats/snow/snocvr_template.yaml.j2 | 4 ++-- 4 files changed, 9 insertions(+), 9 deletions(-) diff --git a/parm/jcb-gdas/algorithm/obstats/snow/ims_snow_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/snow/ims_snow_template.yaml.j2 index cfc9898e0..360bb48c1 100644 --- a/parm/jcb-gdas/algorithm/obstats/snow/ims_snow_template.yaml.j2 +++ b/parm/jcb-gdas/algorithm/obstats/snow/ims_snow_template.yaml.j2 @@ -10,10 +10,10 @@ variables: [totalSnowDepth] groups to process: ['ombg', 'oman'] qc groups: ['EffectiveQC0', 'EffectiveQC1'] - output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_snow.nc" + output file: "{{ stat_current_cycle_YMDH }}.ioda_hofx_stats.{{ obspace }}.nc" regular grid binning: bin size in degrees: 2.0 - use negative longitudes: false + use negative longitudes: false domains to process: - domain: name: "SH" diff --git a/parm/jcb-gdas/algorithm/obstats/snow/madis_snow_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/snow/madis_snow_template.yaml.j2 index e67a9952d..9caabdb52 100644 --- a/parm/jcb-gdas/algorithm/obstats/snow/madis_snow_template.yaml.j2 +++ b/parm/jcb-gdas/algorithm/obstats/snow/madis_snow_template.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: madis_snow + name: madis_snow obsdatain: engine: type: H5File @@ -10,10 +10,10 @@ variables: [totalSnowDepth] groups to process: ['ombg', 'oman'] qc groups: ['EffectiveQC0', 'EffectiveQC1'] - output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_snow.nc" + output file: "{{ stat_current_cycle_YMDH }}.ioda_hofx_stats.{{ obspace }}.nc" regular grid binning: bin size in degrees: 1.0 - use negative longitudes: true + use negative longitudes: true domains to process: - domain: name: "SH" diff --git a/parm/jcb-gdas/algorithm/obstats/snow/sfcsno_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/snow/sfcsno_template.yaml.j2 index 08f5a03ab..cc1c24648 100644 --- a/parm/jcb-gdas/algorithm/obstats/snow/sfcsno_template.yaml.j2 +++ b/parm/jcb-gdas/algorithm/obstats/snow/sfcsno_template.yaml.j2 @@ -10,10 +10,10 @@ variables: [totalSnowDepth] groups to process: ['ombg', 'oman'] qc groups: ['EffectiveQC0', 'EffectiveQC1'] - output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_snow.nc" + output file: "{{ stat_current_cycle_YMDH }}.ioda_hofx_stats.{{ obspace }}.nc" regular grid binning: bin size in degrees: 1.0 - use negative longitudes: true + use negative longitudes: true domains to process: - domain: name: "SH" diff --git a/parm/jcb-gdas/algorithm/obstats/snow/snocvr_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/snow/snocvr_template.yaml.j2 index 47a7a1e08..51035aa83 100644 --- a/parm/jcb-gdas/algorithm/obstats/snow/snocvr_template.yaml.j2 +++ b/parm/jcb-gdas/algorithm/obstats/snow/snocvr_template.yaml.j2 @@ -10,10 +10,10 @@ variables: [totalSnowDepth] groups to process: ['ombg', 'oman'] qc groups: ['EffectiveQC0', 'EffectiveQC1'] - output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_snow.nc" + output file: "{{ stat_current_cycle_YMDH }}.ioda_hofx_stats.{{ obspace }}.nc" regular grid binning: bin size in degrees: 1.0 - use negative longitudes: true + use negative longitudes: true domains to process: - domain: name: "SH" From d3522a4d06a9bef9970b8fb239e5d62988b70f5f Mon Sep 17 00:00:00 2001 From: Jiarui Dong Date: Mon, 3 Nov 2025 14:09:18 -0500 Subject: [PATCH 175/199] Update the commissioning time for new snow observations (#204) This PR updates the commissioning time to enable the use of the newly added snow observations (SNOMAD and SNOCVR). This PR resolves #203 --- parm/jcb-gdas/observation_chronicle/snow/t00z/snocvr.yaml | 2 +- .../jcb-gdas/observation_chronicle/snow/t00z/snocvr_snomad.yaml | 2 +- parm/jcb-gdas/observation_chronicle/snow/t00z/snomad.yaml | 2 +- parm/jcb-gdas/observation_chronicle/snow/t06z/snocvr.yaml | 2 +- .../jcb-gdas/observation_chronicle/snow/t06z/snocvr_snomad.yaml | 2 +- parm/jcb-gdas/observation_chronicle/snow/t06z/snomad.yaml | 2 +- 6 files changed, 6 insertions(+), 6 deletions(-) diff --git a/parm/jcb-gdas/observation_chronicle/snow/t00z/snocvr.yaml b/parm/jcb-gdas/observation_chronicle/snow/t00z/snocvr.yaml index 7d0fde292..0749e56ef 100644 --- a/parm/jcb-gdas/observation_chronicle/snow/t00z/snocvr.yaml +++ b/parm/jcb-gdas/observation_chronicle/snow/t00z/snocvr.yaml @@ -1,6 +1,6 @@ # Instrument metadata # ------------------- -commissioned: 2023-06-01T00:00:00 +commissioned: 2023-05-01T00:00:00 observer_type: conventional # Type of chronicle to use diff --git a/parm/jcb-gdas/observation_chronicle/snow/t00z/snocvr_snomad.yaml b/parm/jcb-gdas/observation_chronicle/snow/t00z/snocvr_snomad.yaml index 6a249526c..41b3ed06c 100644 --- a/parm/jcb-gdas/observation_chronicle/snow/t00z/snocvr_snomad.yaml +++ b/parm/jcb-gdas/observation_chronicle/snow/t00z/snocvr_snomad.yaml @@ -1,6 +1,6 @@ # Instrument metadata # ------------------- -commissioned: 2025-09-01T00:00:00 +commissioned: 2023-05-01T00:00:00 observer_type: conventional # Type of chronicle to use diff --git a/parm/jcb-gdas/observation_chronicle/snow/t00z/snomad.yaml b/parm/jcb-gdas/observation_chronicle/snow/t00z/snomad.yaml index 6a249526c..9db6e445d 100644 --- a/parm/jcb-gdas/observation_chronicle/snow/t00z/snomad.yaml +++ b/parm/jcb-gdas/observation_chronicle/snow/t00z/snomad.yaml @@ -1,6 +1,6 @@ # Instrument metadata # ------------------- -commissioned: 2025-09-01T00:00:00 +commissioned: 2024-11-01T00:00:00 observer_type: conventional # Type of chronicle to use diff --git a/parm/jcb-gdas/observation_chronicle/snow/t06z/snocvr.yaml b/parm/jcb-gdas/observation_chronicle/snow/t06z/snocvr.yaml index e0965b670..0749e56ef 100644 --- a/parm/jcb-gdas/observation_chronicle/snow/t06z/snocvr.yaml +++ b/parm/jcb-gdas/observation_chronicle/snow/t06z/snocvr.yaml @@ -1,6 +1,6 @@ # Instrument metadata # ------------------- -commissioned: 2023-06-01T06:00:00 +commissioned: 2023-05-01T00:00:00 observer_type: conventional # Type of chronicle to use diff --git a/parm/jcb-gdas/observation_chronicle/snow/t06z/snocvr_snomad.yaml b/parm/jcb-gdas/observation_chronicle/snow/t06z/snocvr_snomad.yaml index 6a249526c..41b3ed06c 100644 --- a/parm/jcb-gdas/observation_chronicle/snow/t06z/snocvr_snomad.yaml +++ b/parm/jcb-gdas/observation_chronicle/snow/t06z/snocvr_snomad.yaml @@ -1,6 +1,6 @@ # Instrument metadata # ------------------- -commissioned: 2025-09-01T00:00:00 +commissioned: 2023-05-01T00:00:00 observer_type: conventional # Type of chronicle to use diff --git a/parm/jcb-gdas/observation_chronicle/snow/t06z/snomad.yaml b/parm/jcb-gdas/observation_chronicle/snow/t06z/snomad.yaml index 6a249526c..9db6e445d 100644 --- a/parm/jcb-gdas/observation_chronicle/snow/t06z/snomad.yaml +++ b/parm/jcb-gdas/observation_chronicle/snow/t06z/snomad.yaml @@ -1,6 +1,6 @@ # Instrument metadata # ------------------- -commissioned: 2025-09-01T00:00:00 +commissioned: 2024-11-01T00:00:00 observer_type: conventional # Type of chronicle to use From 4991a0f8064797001080579e391ce278b89d92cb Mon Sep 17 00:00:00 2001 From: DavidNew-NOAA <134300700+DavidNew-NOAA@users.noreply.github.com> Date: Fri, 7 Nov 2025 12:08:07 -0500 Subject: [PATCH 176/199] Remove old obs staging templates (#206) This PR removes obs staging templates that are no longer used. A `stage_observations()` method in the `Jedi` class is being created for https://github.com/NOAA-EMC/global-workflow/issues/4228 that will replicate the functionality of these templates for any model. --- .../algorithm/aero/aero_bias_staging.yaml.j2 | 10 ---------- .../algorithm/aero/aero_obs_staging.yaml.j2 | 8 -------- .../atmosphere/atm_bias_staging.yaml.j2 | 10 ---------- .../atmosphere/atm_obs_staging.yaml.j2 | 8 -------- .../algorithm/marine/soca_obs_staging.yaml.j2 | 9 --------- .../algorithm/snow/snow_obs_staging.yaml.j2 | 18 ------------------ 6 files changed, 63 deletions(-) delete mode 100644 parm/jcb-gdas/algorithm/aero/aero_bias_staging.yaml.j2 delete mode 100644 parm/jcb-gdas/algorithm/aero/aero_obs_staging.yaml.j2 delete mode 100644 parm/jcb-gdas/algorithm/atmosphere/atm_bias_staging.yaml.j2 delete mode 100644 parm/jcb-gdas/algorithm/atmosphere/atm_obs_staging.yaml.j2 delete mode 100644 parm/jcb-gdas/algorithm/marine/soca_obs_staging.yaml.j2 delete mode 100644 parm/jcb-gdas/algorithm/snow/snow_obs_staging.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/aero/aero_bias_staging.yaml.j2 b/parm/jcb-gdas/algorithm/aero/aero_bias_staging.yaml.j2 deleted file mode 100644 index 3a3187ff6..000000000 --- a/parm/jcb-gdas/algorithm/aero/aero_bias_staging.yaml.j2 +++ /dev/null @@ -1,10 +0,0 @@ -mkdir: -- '{{aero_obsbiasout_path}}' -copy: - {% for observation_from_jcb in observations %} - {% if use_observer(observation_from_jcb) %} - {% if observation_from_jcb in bias_files %} - - ['{{aero_obsbiasroot_path}}/{{aero_obsbiasin_prefix}}{{bias_files[observation_from_jcb]}}', '{{aero_obsbiasin_path}}'] - {% endif %} - {% endif %} - {% endfor %} diff --git a/parm/jcb-gdas/algorithm/aero/aero_obs_staging.yaml.j2 b/parm/jcb-gdas/algorithm/aero/aero_obs_staging.yaml.j2 deleted file mode 100644 index 7b704cb21..000000000 --- a/parm/jcb-gdas/algorithm/aero/aero_obs_staging.yaml.j2 +++ /dev/null @@ -1,8 +0,0 @@ -mkdir: - - '{{aero_obsdatain_path}}' -copy_opt: - {% for observation_from_jcb in observations %} - {% if use_observer(observation_from_jcb) %} - - ['{{aero_obsdataroot_path}}/{{aero_obsdatain_prefix}}{{observation_from_jcb}}{{aero_obsdatain_suffix}}', '{{aero_obsdatain_path}}'] - {% endif %} - {% endfor %} diff --git a/parm/jcb-gdas/algorithm/atmosphere/atm_bias_staging.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/atm_bias_staging.yaml.j2 deleted file mode 100644 index c91ea2e80..000000000 --- a/parm/jcb-gdas/algorithm/atmosphere/atm_bias_staging.yaml.j2 +++ /dev/null @@ -1,10 +0,0 @@ -mkdir: - - '{{atmosphere_obsbiasout_path}}' -copy: - {% for observation_from_jcb in observations %} - {% if use_observer(observation_from_jcb) %} - {% if observation_from_jcb in bias_files %} - - ['{{atmosphere_obsbiasroot_path}}/{{atmosphere_obsbiasin_prefix}}{{bias_files[observation_from_jcb]}}', '{{atmosphere_obsbiasin_path}}'] - {% endif %} - {% endif %} - {% endfor %} diff --git a/parm/jcb-gdas/algorithm/atmosphere/atm_obs_staging.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/atm_obs_staging.yaml.j2 deleted file mode 100644 index 34d098490..000000000 --- a/parm/jcb-gdas/algorithm/atmosphere/atm_obs_staging.yaml.j2 +++ /dev/null @@ -1,8 +0,0 @@ -mkdir: - - '{{atmosphere_obsdatain_path}}' -copy: - {% for observation_from_jcb in observations %} - {% if use_observer(observation_from_jcb) %} - - ['{{atmosphere_obsdataroot_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}', '{{atmosphere_obsdatain_path}}'] - {% endif %} - {% endfor %} diff --git a/parm/jcb-gdas/algorithm/marine/soca_obs_staging.yaml.j2 b/parm/jcb-gdas/algorithm/marine/soca_obs_staging.yaml.j2 deleted file mode 100644 index 49b972bd1..000000000 --- a/parm/jcb-gdas/algorithm/marine/soca_obs_staging.yaml.j2 +++ /dev/null @@ -1,9 +0,0 @@ -mkdir: - - '{{marine_obsdatain_path}}' - - '{{marine_obsdataout_path}}' -copy_opt: - {% for observation_from_jcb in observations %} - {% if use_observer(observation_from_jcb) %} - - ['{{marine_obsdataroot_path}}/{{marine_obsdatain_prefix}}{{observation_from_jcb}}{{marine_obsdatain_suffix}}', '{{marine_obsdatain_path}}'] - {% endif %} - {% endfor %} diff --git a/parm/jcb-gdas/algorithm/snow/snow_obs_staging.yaml.j2 b/parm/jcb-gdas/algorithm/snow/snow_obs_staging.yaml.j2 deleted file mode 100644 index 17703ebd2..000000000 --- a/parm/jcb-gdas/algorithm/snow/snow_obs_staging.yaml.j2 +++ /dev/null @@ -1,18 +0,0 @@ -mkdir: - - '{{snow_obsdatain_path}}' -copy_opt: - {% for observation_from_jcb in observations %} - {% if use_observer(observation_from_jcb) %} - {% if not observation_from_jcb == 'ims_snow' %} - # skip if ims_snow since it is special - {% if observation_from_jcb == 'madis_snow' %} - # madis_snow is a .nc file with filename 'snocvr_snow' in global GDA. - - ['{{snow_obsdataroot_path}}/{{snow_obsdatain_prefix}}snocvr_snow.nc4', '{{snow_obsdatain_path}}'] - {% elif observation_from_jcb == 'ghcn_snow' %} - - ['{{snow_obsdataroot_path}}/{{snow_obsdatain_prefix}}ghcn_snow.nc', '{{snow_obsdatain_path}}'] - {% else %} - - ['{{snow_obsdataroot_path}}/{{snow_obsdatain_prefix}}{{observation_from_jcb}}{{snow_obsdatain_suffix}}', '{{snow_obsdatain_path}}'] - {% endif %} - {% endif %} - {% endif %} - {% endfor %} From 753b445095acb485f543ab9e1e753f6769c3e950 Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Wed, 12 Nov 2025 13:47:26 -0500 Subject: [PATCH 177/199] Use ObsForge for AOD (#205) This PR renames the AOD obsspaces to match the expected ObsForge naming convention. --- .../{viirs_n20_aod.yaml.j2 => retrieval_aod_viirs_n20.yaml.j2} | 2 +- .../{viirs_n21_aod.yaml.j2 => retrieval_aod_viirs_n21.yaml.j2} | 2 +- .../{viirs_npp_aod.yaml.j2 => retrieval_aod_viirs_npp.yaml.j2} | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) rename parm/jcb-gdas/observations/aero/{viirs_n20_aod.yaml.j2 => retrieval_aod_viirs_n20.yaml.j2} (97%) rename parm/jcb-gdas/observations/aero/{viirs_n21_aod.yaml.j2 => retrieval_aod_viirs_n21.yaml.j2} (97%) rename parm/jcb-gdas/observations/aero/{viirs_npp_aod.yaml.j2 => retrieval_aod_viirs_npp.yaml.j2} (97%) diff --git a/parm/jcb-gdas/observations/aero/viirs_n20_aod.yaml.j2 b/parm/jcb-gdas/observations/aero/retrieval_aod_viirs_n20.yaml.j2 similarity index 97% rename from parm/jcb-gdas/observations/aero/viirs_n20_aod.yaml.j2 rename to parm/jcb-gdas/observations/aero/retrieval_aod_viirs_n20.yaml.j2 index 06613f4a4..9b3c9a7b6 100644 --- a/parm/jcb-gdas/observations/aero/viirs_n20_aod.yaml.j2 +++ b/parm/jcb-gdas/observations/aero/retrieval_aod_viirs_n20.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: viirs_n20_aod + name: retrieval_aod_viirs_n20 obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/aero/viirs_n21_aod.yaml.j2 b/parm/jcb-gdas/observations/aero/retrieval_aod_viirs_n21.yaml.j2 similarity index 97% rename from parm/jcb-gdas/observations/aero/viirs_n21_aod.yaml.j2 rename to parm/jcb-gdas/observations/aero/retrieval_aod_viirs_n21.yaml.j2 index 87119ad52..be8475a21 100644 --- a/parm/jcb-gdas/observations/aero/viirs_n21_aod.yaml.j2 +++ b/parm/jcb-gdas/observations/aero/retrieval_aod_viirs_n21.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: viirs_n21_aod + name: retrieval_aod_viirs_n21 obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/aero/viirs_npp_aod.yaml.j2 b/parm/jcb-gdas/observations/aero/retrieval_aod_viirs_npp.yaml.j2 similarity index 97% rename from parm/jcb-gdas/observations/aero/viirs_npp_aod.yaml.j2 rename to parm/jcb-gdas/observations/aero/retrieval_aod_viirs_npp.yaml.j2 index fc28d42e6..831eea4ae 100644 --- a/parm/jcb-gdas/observations/aero/viirs_npp_aod.yaml.j2 +++ b/parm/jcb-gdas/observations/aero/retrieval_aod_viirs_npp.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: viirs_npp_aod + name: retrieval_aod_viirs_npp obsdatain: engine: type: H5File From e8ee211fa40d64248368bdaac3861aea8c246803 Mon Sep 17 00:00:00 2001 From: DavidNew-NOAA <134300700+DavidNew-NOAA@users.noreply.github.com> Date: Thu, 13 Nov 2025 10:56:33 -0500 Subject: [PATCH 178/199] Put quotes around datetimes (#209) This PR puts quotes around dates that are not already quoted. This prevents a rare bug that can cause the date formatting of a JCB date string variable to change. --- parm/jcb-gdas/algorithm/aero/aero_addincrement.yaml.j2 | 2 +- parm/jcb-gdas/algorithm/atmosphere/atm_addincrement.yaml.j2 | 2 +- .../algorithm/atmosphere/fv3jedi_correction_increment.yaml.j2 | 2 +- .../algorithm/atmosphere/fv3jedi_ensemble_recenter.yaml.j2 | 2 +- parm/jcb-gdas/algorithm/snow/snow_addincrement.yaml.j2 | 2 +- 5 files changed, 5 insertions(+), 5 deletions(-) diff --git a/parm/jcb-gdas/algorithm/aero/aero_addincrement.yaml.j2 b/parm/jcb-gdas/algorithm/aero/aero_addincrement.yaml.j2 index b950f5cb1..895995ee4 100644 --- a/parm/jcb-gdas/algorithm/aero/aero_addincrement.yaml.j2 +++ b/parm/jcb-gdas/algorithm/aero/aero_addincrement.yaml.j2 @@ -23,7 +23,7 @@ increment geometry: npz: {{ aero_npz_ges }} field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_restart.yaml state: - datetime: {{ aero_background_time_iso }} + datetime: '{{ aero_background_time_iso }}' state variables: {{ analysis_variables }} filetype: cube sphere history provider: ufs diff --git a/parm/jcb-gdas/algorithm/atmosphere/atm_addincrement.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/atm_addincrement.yaml.j2 index 4d6a8f410..b345f75d0 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/atm_addincrement.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/atm_addincrement.yaml.j2 @@ -23,7 +23,7 @@ increment geometry: npz: {{ atmosphere_npz_ges }} field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_fv3inc.yaml state: - datetime: {{ atmosphere_background_time_iso }} + datetime: '{{ atmosphere_background_time_iso }}' state variables: - eastward_wind - northward_wind diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_correction_increment.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_correction_increment.yaml.j2 index 200897491..1c135b55a 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_correction_increment.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_correction_increment.yaml.j2 @@ -54,7 +54,7 @@ correction increment geometry: npz: {{ atmosphere_npz_anl }} forecast hours: {% for ihour in range( atmosphere_iau_hours | length ) %} -- datetime: {{ atmosphere_iau_times_iso[ihour] }} +- datetime: '{{ atmosphere_iau_times_iso[ihour] }}' deterministic background: filetype: cube sphere history provider: ufs diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_ensemble_recenter.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_ensemble_recenter.yaml.j2 index baa6a005a..f68e2e106 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_ensemble_recenter.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_ensemble_recenter.yaml.j2 @@ -24,7 +24,7 @@ members: npy: {{ atmosphere_npy_anl }} npz: {{ atmosphere_npz_anl }} state: - datetime: {{ atmosphere_iau_times_iso[ihour] }} + datetime: '{{ atmosphere_iau_times_iso[ihour] }}' state variables: - eastward_wind - northward_wind diff --git a/parm/jcb-gdas/algorithm/snow/snow_addincrement.yaml.j2 b/parm/jcb-gdas/algorithm/snow/snow_addincrement.yaml.j2 index 96288640b..1ac692e0d 100644 --- a/parm/jcb-gdas/algorithm/snow/snow_addincrement.yaml.j2 +++ b/parm/jcb-gdas/algorithm/snow/snow_addincrement.yaml.j2 @@ -23,7 +23,7 @@ increment geometry: npz: {{ snow_npz_ges }} field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_restart.yaml state: - datetime: {{ snow_background_time_iso }} + datetime: '{{ snow_background_time_iso }}' state variables: {{ analysis_variables }} filetype: cube sphere history provider: ufs From cb61e7a881ec65ee8ecba96e84e7f5e2aa418180 Mon Sep 17 00:00:00 2001 From: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com> Date: Mon, 24 Nov 2025 11:50:02 -0500 Subject: [PATCH 179/199] Extend templates to run atmospheric JEDI EnVar DA with GSI EnKF (#208) --- .../atmosphere/atm_addincrement.yaml.j2 | 6 +++--- .../fv3jedi_correction_increment.yaml.j2 | 6 +++--- .../fv3jedi_fv3inc_variational.yaml.j2 | 18 ++++++++++++------ ...> retrieval_aod_viirs_n20_template.yaml.j2} | 2 +- ...> retrieval_aod_viirs_n21_template.yaml.j2} | 2 +- ...> retrieval_aod_viirs_npp_template.yaml.j2} | 2 +- .../atmosphere_3dvar_outer_loop_1.yaml.j2 | 4 ++-- .../atmosphere_3dvar_outer_loop_2.yaml.j2 | 4 ++-- ...sphere_final_increment_cubed_sphere.yaml.j2 | 6 +++--- ...atmosphere_final_increment_gaussian.yaml.j2 | 6 +++--- .../atmosphere/prepbufr_adpsfc.yaml.j2 | 2 +- .../atmosphere/prepbufr_adpupa.yaml.j2 | 16 ++++++++-------- .../atmosphere/prepbufr_sfcshp.yaml.j2 | 2 +- .../gdas-atmosphere-templates.yaml | 10 ++++++++++ 14 files changed, 51 insertions(+), 35 deletions(-) rename parm/jcb-gdas/algorithm/obstats/aero/{viirs_n20_template.yaml.j2 => retrieval_aod_viirs_n20_template.yaml.j2} (97%) rename parm/jcb-gdas/algorithm/obstats/aero/{viirs_n21_template.yaml.j2 => retrieval_aod_viirs_n21_template.yaml.j2} (97%) rename parm/jcb-gdas/algorithm/obstats/aero/{viirs_npp_template.yaml.j2 => retrieval_aod_viirs_npp_template.yaml.j2} (97%) diff --git a/parm/jcb-gdas/algorithm/atmosphere/atm_addincrement.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/atm_addincrement.yaml.j2 index b345f75d0..7204db61a 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/atm_addincrement.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/atm_addincrement.yaml.j2 @@ -18,9 +18,9 @@ increment geometry: layout: - {{ atmosphere_layout_x }} - {{ atmosphere_layout_y }} - npx: {{ atmosphere_npx_ges }} - npy: {{ atmosphere_npy_ges }} - npz: {{ atmosphere_npz_ges }} + npx: {{ atmosphere_npx_anl }} + npy: {{ atmosphere_npy_anl }} + npz: {{ atmosphere_npz_anl }} field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_fv3inc.yaml state: datetime: '{{ atmosphere_background_time_iso }}' diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_correction_increment.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_correction_increment.yaml.j2 index 1c135b55a..1f996bacc 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_correction_increment.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_correction_increment.yaml.j2 @@ -27,9 +27,9 @@ variational increment geometry: layout: - {{ atmosphere_layout_x }} - {{ atmosphere_layout_y }} - npx: {{ atmosphere_npx_ges }} - npy: {{ atmosphere_npy_ges }} - npz: {{ atmosphere_npz_ges }} + npx: {{ atmosphere_npx_anl }} + npy: {{ atmosphere_npy_anl }} + npz: {{ atmosphere_npz_anl }} ensemble mean analysis geometry: fms initialization: namelist filename: ./fv3jedi/fmsmpp.nml diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 index 8280f54e7..97e44406b 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 @@ -49,9 +49,9 @@ jedi increment geometry: layout: - {{ atmosphere_layout_x }} - {{ atmosphere_layout_y }} - npx: {{ atmosphere_npx_ges }} - npy: {{ atmosphere_npy_ges }} - npz: {{ atmosphere_npz_ges }} + npx: {{ atmosphere_npx_his }} + npy: {{ atmosphere_npy_his }} + npz: {{ atmosphere_npz_his }} fv3 increment geometry: fms initialization: namelist filename: ./fv3jedi/fmsmpp.nml @@ -60,9 +60,9 @@ fv3 increment geometry: layout: - {{ atmosphere_layout_x }} - {{ atmosphere_layout_y }} - npx: {{ atmosphere_npx_ges }} - npy: {{ atmosphere_npy_ges }} - npz: {{ atmosphere_npz_ges }} + npx: {{ atmosphere_npx_his }} + npy: {{ atmosphere_npy_his }} + npz: {{ atmosphere_npz_his }} members: - background input: datapath: ./bkg @@ -93,10 +93,16 @@ members: datapath: ./anl filename: cubed_sphere_grid_atminc.jedi.nc fv3 increment output: +{% if atmosphere_variational_increment_type == 'gaussian' %} + filetype: auxgrid + gridtype: gaussian + filename: ./anl/atminc. +{% else %} filetype: fms restart is restart: false datapath: ./anl filename_nonrestart: {{ atmosphere_variational_analysis_prefix }}cubed_sphere_grid_atminc.nc +{% endif %} field io names: eastward_wind: u_inc northward_wind: v_inc diff --git a/parm/jcb-gdas/algorithm/obstats/aero/viirs_n20_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/aero/retrieval_aod_viirs_n20_template.yaml.j2 similarity index 97% rename from parm/jcb-gdas/algorithm/obstats/aero/viirs_n20_template.yaml.j2 rename to parm/jcb-gdas/algorithm/obstats/aero/retrieval_aod_viirs_n20_template.yaml.j2 index b62a7c66b..3bed75d11 100644 --- a/parm/jcb-gdas/algorithm/obstats/aero/viirs_n20_template.yaml.j2 +++ b/parm/jcb-gdas/algorithm/obstats/aero/retrieval_aod_viirs_n20_template.yaml.j2 @@ -3,7 +3,7 @@ obsdatain: engine: type: H5File - obsfile: {{ aero_obsdatain_path }}/aero/diag_{{ obspace }}_aod_{{ stat_current_cycle_YMDH }}.nc + obsfile: {{ aero_obsdatain_path }}/aero/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc simulated variables: ['aerosolOpticalDepth'] observed variables: ['aerosolOpticalDepth'] variables: ['aerosolOpticalDepth'] diff --git a/parm/jcb-gdas/algorithm/obstats/aero/viirs_n21_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/aero/retrieval_aod_viirs_n21_template.yaml.j2 similarity index 97% rename from parm/jcb-gdas/algorithm/obstats/aero/viirs_n21_template.yaml.j2 rename to parm/jcb-gdas/algorithm/obstats/aero/retrieval_aod_viirs_n21_template.yaml.j2 index e03a21981..fc471d35e 100644 --- a/parm/jcb-gdas/algorithm/obstats/aero/viirs_n21_template.yaml.j2 +++ b/parm/jcb-gdas/algorithm/obstats/aero/retrieval_aod_viirs_n21_template.yaml.j2 @@ -3,7 +3,7 @@ obsdatain: engine: type: H5File - obsfile: {{ aero_obsdatain_path }}/aero/diag_{{ obspace }}_aod_{{ stat_current_cycle_YMDH }}.nc + obsfile: {{ aero_obsdatain_path }}/aero/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc simulated variables: ['aerosolOpticalDepth'] observed variables: ['aerosolOpticalDepth'] variables: ['aerosolOpticalDepth'] diff --git a/parm/jcb-gdas/algorithm/obstats/aero/viirs_npp_template.yaml.j2 b/parm/jcb-gdas/algorithm/obstats/aero/retrieval_aod_viirs_npp_template.yaml.j2 similarity index 97% rename from parm/jcb-gdas/algorithm/obstats/aero/viirs_npp_template.yaml.j2 rename to parm/jcb-gdas/algorithm/obstats/aero/retrieval_aod_viirs_npp_template.yaml.j2 index 821769501..31121e686 100644 --- a/parm/jcb-gdas/algorithm/obstats/aero/viirs_npp_template.yaml.j2 +++ b/parm/jcb-gdas/algorithm/obstats/aero/retrieval_aod_viirs_npp_template.yaml.j2 @@ -3,7 +3,7 @@ obsdatain: engine: type: H5File - obsfile: {{ aero_obsdatain_path }}/aero/diag_{{ obspace }}_aod_{{ stat_current_cycle_YMDH }}.nc + obsfile: {{ aero_obsdatain_path }}/aero/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc simulated variables: ['aerosolOpticalDepth'] observed variables: ['aerosolOpticalDepth'] variables: ['aerosolOpticalDepth'] diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 index 3b2c58cb5..bcb4a4600 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_1.yaml.j2 @@ -1,5 +1,5 @@ -- ninner: 2 - gradient norm reduction: 1e-10 +- ninner: {{atmosphere_ninner_1}} + gradient norm reduction: {{atmosphere_grad_red_1}} test: on geometry: fms initialization: diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 index 93de3f1ba..ab5b15d22 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_3dvar_outer_loop_2.yaml.j2 @@ -1,5 +1,5 @@ -- ninner: 4 - gradient norm reduction: 1e-10 +- ninner: {{atmosphere_ninner_2}} + gradient norm reduction: {{atmosphere_grad_red_2}} test: on geometry: fms initialization: diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_cubed_sphere.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_cubed_sphere.yaml.j2 index 9990e7a4a..0f902d716 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_cubed_sphere.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_cubed_sphere.yaml.j2 @@ -29,6 +29,6 @@ geometry: layout: - {{atmosphere_layout_x}} - {{atmosphere_layout_y}} - npx: {{atmosphere_npx_ges}} - npy: {{atmosphere_npy_ges}} - npz: {{atmosphere_npz_ges}} + npx: {{atmosphere_npx_his}} + npy: {{atmosphere_npy_his}} + npz: {{atmosphere_npz_his}} diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_gaussian.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_gaussian.yaml.j2 index 414aa0bd6..ae88b090f 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_gaussian.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_gaussian.yaml.j2 @@ -20,6 +20,6 @@ geometry: layout: - {{atmosphere_layout_x}} - {{atmosphere_layout_y}} - npx: {{atmosphere_npx_anl}} - npy: {{atmosphere_npy_anl}} - npz: {{atmosphere_npz_anl}} + npx: {{atmosphere_npx_his}} + npy: {{atmosphere_npy_his}} + npz: {{atmosphere_npz_his}} diff --git a/parm/jcb-gdas/observations/atmosphere/prepbufr_adpsfc.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/prepbufr_adpsfc.yaml.j2 index 584c548a0..c68693a24 100644 --- a/parm/jcb-gdas/observations/atmosphere/prepbufr_adpsfc.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/prepbufr_adpsfc.yaml.j2 @@ -252,7 +252,7 @@ name: ObsType/stationPressure is_in: 180 - variable: - name: ObsSubType/stationPressure + name: MetaData/observationSubTypeNum is_in: 0 action: name: inflate error diff --git a/parm/jcb-gdas/observations/atmosphere/prepbufr_adpupa.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/prepbufr_adpupa.yaml.j2 index 584c548a0..91c7f9b9a 100644 --- a/parm/jcb-gdas/observations/atmosphere/prepbufr_adpupa.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/prepbufr_adpupa.yaml.j2 @@ -43,7 +43,7 @@ filter variables: - name: stationPressure where: - - variable: ObsType/stationPressure + - variable: ObsType/pressure is_in: [181] action: name: assign error @@ -60,7 +60,7 @@ filter variables: - name: stationPressure where: - - variable: ObsType/stationPressure + - variable: ObsType/pressure is_in: [187] action: name: assign error @@ -78,7 +78,7 @@ filter variables: - name: stationPressure where: - - variable: ObsType/stationPressure + - variable: ObsType/pressure is_in: [180] action: name: assign error @@ -95,7 +95,7 @@ filter variables: - name: stationPressure where: - - variable: ObsType/stationPressure + - variable: ObsType/pressure is_in: [183] action: name: assign error @@ -106,7 +106,7 @@ filter variables: - name: stationPressure where: - - variable: ObsType/stationPressure + - variable: ObsType/pressure is_in: [120] action: name: assign error @@ -154,7 +154,7 @@ - filter: Variable Assignment where: - variable: - name: ObsType/stationPressure + name: ObsType/pressure is_in: 183 assignments: - name: PreUseFlag/stationPressure @@ -203,7 +203,7 @@ #- filter: RejectList # where: # - variable: - # name: ObsType/stationPressure + # name: ObsType/pressure # is_in: 183 ## Reject surface pressure below 500 hPa #- filter: Bounds Check @@ -249,7 +249,7 @@ - name: stationPressure where: - variable: - name: ObsType/stationPressure + name: ObsType/pressure is_in: 180 - variable: name: ObsSubType/stationPressure diff --git a/parm/jcb-gdas/observations/atmosphere/prepbufr_sfcshp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/prepbufr_sfcshp.yaml.j2 index d00e23eb9..c68693a24 100644 --- a/parm/jcb-gdas/observations/atmosphere/prepbufr_sfcshp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/prepbufr_sfcshp.yaml.j2 @@ -252,7 +252,7 @@ name: ObsType/stationPressure is_in: 180 - variable: - ObsSubType/stationPressure + name: MetaData/observationSubTypeNum is_in: 0 action: name: inflate error diff --git a/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml b/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml index f6a8dd11c..3ceef304a 100644 --- a/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml +++ b/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml @@ -51,6 +51,9 @@ atmosphere_layout_y: 2 atmosphere_npx_ges: 361 atmosphere_npy_ges: 361 atmosphere_npz_ges: 127 +atmosphere_npx_his: 361 +atmosphere_npy_his: 361 +atmosphere_npz_his: 127 atmosphere_npx_anl: 361 atmosphere_npy_anl: 361 atmosphere_npz_anl: 127 @@ -75,11 +78,18 @@ atmosphere_number_ensemble_members: 3 atmosphere_layout_gsib_x: 2 atmosphere_layout_gsib_y: 2 +# Minimization options +atmosphere_ninner_1: 4 +atmosphere_ninner_2: 2 +atmosphere_grad_red_1: 1e-5 +atmosphere_grad_red_2: 1e-5 + # Forecasting atmosphere_forecast_length: PT6H atmosphere_forecast_timestep: PT1H # Write final increment on Gaussian grid in variational +atmosphere_variational_increment_type: "gaussian" atmosphere_final_increment_prefix: "./anl/atminc." # Observation things From d6379963189ed0085930af50cfa8c9b6ac2786c8 Mon Sep 17 00:00:00 2001 From: Yaping Wang <49168260+ypwang19@users.noreply.github.com> Date: Tue, 2 Dec 2025 08:58:10 -0500 Subject: [PATCH 180/199] Corrects sea salt bin mapping between forward operator and model (#211) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This PR corrects sea salt bin mapping between CRTM forward operator and model. Sea-salt bins 2–5 instead of bins 1–4 from model should be used to compute hofx, and be updated after DA. This PR updates the sea salt variable names accordingly. --------- Co-authored-by: ypwang19 --- .../algorithm/aero/aero_convert_background.yaml.j2 | 1 + .../algorithm/aero/aero_gen_bmatrix_diagb.yaml.j2 | 2 ++ .../algorithm/aero/aero_gen_bmatrix_diffusion.yaml.j2 | 1 + .../aero/aero_gen_bmatrix_rescale_default.yaml.j2 | 5 ++++- parm/jcb-gdas/model/aero/aero_background.yaml.j2 | 10 ++++------ .../aero_background_error_static_diffusion.yaml.j2 | 9 ++++----- .../aero/aero_final_increment_cubed_sphere.yaml.j2 | 8 ++++---- parm/jcb-gdas/model/aero/aero_model_pseudo.yaml.j2 | 10 ++++------ 8 files changed, 24 insertions(+), 22 deletions(-) diff --git a/parm/jcb-gdas/algorithm/aero/aero_convert_background.yaml.j2 b/parm/jcb-gdas/algorithm/aero/aero_convert_background.yaml.j2 index d3f570b87..7b4d38c0e 100644 --- a/parm/jcb-gdas/algorithm/aero/aero_convert_background.yaml.j2 +++ b/parm/jcb-gdas/algorithm/aero/aero_convert_background.yaml.j2 @@ -39,6 +39,7 @@ states: - mass_fraction_of_sea_salt002_in_air - mass_fraction_of_sea_salt003_in_air - mass_fraction_of_sea_salt004_in_air + - mass_fraction_of_sea_salt005_in_air field io names: mass_fraction_of_dust001_in_air: dust1 mass_fraction_of_dust002_in_air: dust2 diff --git a/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diagb.yaml.j2 b/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diagb.yaml.j2 index 045991aed..62e88a91a 100644 --- a/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diagb.yaml.j2 +++ b/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diagb.yaml.j2 @@ -40,6 +40,7 @@ background: - mass_fraction_of_sea_salt002_in_air - mass_fraction_of_sea_salt003_in_air - mass_fraction_of_sea_salt004_in_air + - mass_fraction_of_sea_salt005_in_air field io names: mass_fraction_of_dust001_in_air: dust1 mass_fraction_of_dust002_in_air: dust2 @@ -134,6 +135,7 @@ variables: - mass_fraction_of_sea_salt002_in_air - mass_fraction_of_sea_salt003_in_air - mass_fraction_of_sea_salt004_in_air + - mass_fraction_of_sea_salt005_in_air {% set rescaling_factor_file = aero_rescaling_factor_file|default(aero_gen_bmatrix_rescale_default) %} {% include rescaling_factor_file %} diff --git a/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diffusion.yaml.j2 b/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diffusion.yaml.j2 index 9742aaf90..d1eaa28e2 100644 --- a/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diffusion.yaml.j2 +++ b/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_diffusion.yaml.j2 @@ -28,6 +28,7 @@ background: - mass_fraction_of_sea_salt002_in_air - mass_fraction_of_sea_salt003_in_air - mass_fraction_of_sea_salt004_in_air + - mass_fraction_of_sea_salt005_in_air field io names: mass_fraction_of_dust001_in_air: dust1 mass_fraction_of_dust002_in_air: dust2 diff --git a/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_rescale_default.yaml.j2 b/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_rescale_default.yaml.j2 index b5f80ed8a..55d14aebf 100644 --- a/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_rescale_default.yaml.j2 +++ b/parm/jcb-gdas/algorithm/aero/aero_gen_bmatrix_rescale_default.yaml.j2 @@ -1,6 +1,6 @@ rescaling factors: mass_fraction_of_sulfate_in_air: - rescaling factor: 1.0 + rescaling factor: 0.1 mass_fraction_of_hydrophobic_black_carbon_in_air: ocean rescaling factor: 1.0 land rescaling factor: 1.0 @@ -25,6 +25,9 @@ rescaling factors: mass_fraction_of_sea_salt004_in_air: ocean rescaling factor: 1.0 land rescaling factor: 1.0 + mass_fraction_of_sea_salt005_in_air: + ocean rescaling factor: 1.0 + land rescaling factor: 1.0 mass_fraction_of_dust001_in_air: rescaling factor: 1.0 mass_fraction_of_dust002_in_air: diff --git a/parm/jcb-gdas/model/aero/aero_background.yaml.j2 b/parm/jcb-gdas/model/aero/aero_background.yaml.j2 index 85e796a24..daf6d4ef8 100644 --- a/parm/jcb-gdas/model/aero/aero_background.yaml.j2 +++ b/parm/jcb-gdas/model/aero/aero_background.yaml.j2 @@ -23,7 +23,6 @@ state variables: - mass_fraction_of_sea_salt002_in_air - mass_fraction_of_sea_salt003_in_air - mass_fraction_of_sea_salt004_in_air -- mass_fraction_of_sea_salt005_in_air field io names: air_temperature: T air_pressure_thickness: delp @@ -38,8 +37,7 @@ field io names: mass_fraction_of_dust003_in_air: dust3 mass_fraction_of_dust004_in_air: dust4 mass_fraction_of_dust005_in_air: dust5 - mass_fraction_of_sea_salt001_in_air: seas1 - mass_fraction_of_sea_salt002_in_air: seas2 - mass_fraction_of_sea_salt003_in_air: seas3 - mass_fraction_of_sea_salt004_in_air: seas4 - mass_fraction_of_sea_salt005_in_air: seas5 + mass_fraction_of_sea_salt001_in_air: seas2 + mass_fraction_of_sea_salt002_in_air: seas3 + mass_fraction_of_sea_salt003_in_air: seas4 + mass_fraction_of_sea_salt004_in_air: seas5 diff --git a/parm/jcb-gdas/model/aero/aero_background_error_static_diffusion.yaml.j2 b/parm/jcb-gdas/model/aero/aero_background_error_static_diffusion.yaml.j2 index 02de0dfa4..cb29115f3 100644 --- a/parm/jcb-gdas/model/aero/aero_background_error_static_diffusion.yaml.j2 +++ b/parm/jcb-gdas/model/aero/aero_background_error_static_diffusion.yaml.j2 @@ -42,11 +42,10 @@ saber outer blocks: mass_fraction_of_dust003_in_air: dust3 mass_fraction_of_dust004_in_air: dust4 mass_fraction_of_dust005_in_air: dust5 - mass_fraction_of_sea_salt001_in_air: seas1 - mass_fraction_of_sea_salt002_in_air: seas2 - mass_fraction_of_sea_salt003_in_air: seas3 - mass_fraction_of_sea_salt004_in_air: seas4 - mass_fraction_of_sea_salt005_in_air: seas5 + mass_fraction_of_sea_salt001_in_air: seas2 + mass_fraction_of_sea_salt002_in_air: seas3 + mass_fraction_of_sea_salt003_in_air: seas4 + mass_fraction_of_sea_salt004_in_air: seas5 mass_fraction_of_hydrophobic_black_carbon_in_air: bc1 mass_fraction_of_hydrophilic_black_carbon_in_air: bc2 mass_fraction_of_hydrophobic_organic_carbon_in_air: oc1 diff --git a/parm/jcb-gdas/model/aero/aero_final_increment_cubed_sphere.yaml.j2 b/parm/jcb-gdas/model/aero/aero_final_increment_cubed_sphere.yaml.j2 index 9b3ebe318..74c96986b 100644 --- a/parm/jcb-gdas/model/aero/aero_final_increment_cubed_sphere.yaml.j2 +++ b/parm/jcb-gdas/model/aero/aero_final_increment_cubed_sphere.yaml.j2 @@ -44,10 +44,10 @@ output: mass_fraction_of_dust003_in_air: dust3 mass_fraction_of_dust004_in_air: dust4 mass_fraction_of_dust005_in_air: dust5 - mass_fraction_of_sea_salt001_in_air: seas1 - mass_fraction_of_sea_salt002_in_air: seas2 - mass_fraction_of_sea_salt003_in_air: seas3 - mass_fraction_of_sea_salt004_in_air: seas4 + mass_fraction_of_sea_salt001_in_air: seas2 + mass_fraction_of_sea_salt002_in_air: seas3 + mass_fraction_of_sea_salt003_in_air: seas4 + mass_fraction_of_sea_salt004_in_air: seas5 mass_fraction_of_hydrophobic_black_carbon_in_air: bc1 mass_fraction_of_hydrophilic_black_carbon_in_air: bc2 mass_fraction_of_hydrophobic_organic_carbon_in_air: oc1 diff --git a/parm/jcb-gdas/model/aero/aero_model_pseudo.yaml.j2 b/parm/jcb-gdas/model/aero/aero_model_pseudo.yaml.j2 index 149db9181..a5c879a5b 100644 --- a/parm/jcb-gdas/model/aero/aero_model_pseudo.yaml.j2 +++ b/parm/jcb-gdas/model/aero/aero_model_pseudo.yaml.j2 @@ -25,7 +25,6 @@ state variables: - mass_fraction_of_sea_salt002_in_air - mass_fraction_of_sea_salt003_in_air - mass_fraction_of_sea_salt004_in_air -- mass_fraction_of_sea_salt005_in_air field io names: air_temperature: T air_pressure_thickness: delp @@ -40,9 +39,8 @@ field io names: mass_fraction_of_dust003_in_air: dust3 mass_fraction_of_dust004_in_air: dust4 mass_fraction_of_dust005_in_air: dust5 - mass_fraction_of_sea_salt001_in_air: seas1 - mass_fraction_of_sea_salt002_in_air: seas2 - mass_fraction_of_sea_salt003_in_air: seas3 - mass_fraction_of_sea_salt004_in_air: seas4 - mass_fraction_of_sea_salt005_in_air: seas5 + mass_fraction_of_sea_salt001_in_air: seas2 + mass_fraction_of_sea_salt002_in_air: seas3 + mass_fraction_of_sea_salt003_in_air: seas4 + mass_fraction_of_sea_salt004_in_air: seas5 tstep: {{aero_forecast_timestep}} From f93b4d3b692a7af9d24d887c082ff84c1b0493ad Mon Sep 17 00:00:00 2001 From: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com> Date: Thu, 11 Dec 2025 13:48:50 -0500 Subject: [PATCH 181/199] Add select hydrometeors to atmospheric DA yamls (#216) This PR adds `snow_water`, `rain_water`, and `graupel` to atmospheric DA yamls. An additional change is to turn on fov averaging for atms_n20 in `observations/atmosphere/atms_n20.yaml.j2`. Resolves #215 --- .../atmosphere/atm_addincrement.yaml.j2 | 15 ++++++++++++ .../fv3jedi_correction_increment.yaml.j2 | 12 ++++++++++ .../fv3jedi_ensemble_recenter.yaml.j2 | 15 ++++++++++++ .../atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 | 15 ++++++++++++ .../fv3jedi_fv3inc_variational.yaml.j2 | 15 ++++++++++++ ...ackground_error_hybrid_gsibec_bump.yaml.j2 | 23 ++++++++++++++++++- ...phere_final_increment_cubed_sphere.yaml.j2 | 6 +++++ ...t_ensemble_increments_cubed_sphere.yaml.j2 | 3 +++ .../observations/atmosphere/atms_n20.yaml.j2 | 2 ++ 9 files changed, 105 insertions(+), 1 deletion(-) diff --git a/parm/jcb-gdas/algorithm/atmosphere/atm_addincrement.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/atm_addincrement.yaml.j2 index 7204db61a..b11b3bdfc 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/atm_addincrement.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/atm_addincrement.yaml.j2 @@ -30,6 +30,9 @@ state: - water_vapor_mixing_ratio_wrt_moist_air - cloud_liquid_ice - cloud_liquid_water + - snow_water + - rain_water + - graupel - ozone_mass_mixing_ratio - air_pressure_thickness - layer_thickness @@ -40,6 +43,9 @@ state: water_vapor_mixing_ratio_wrt_moist_air: spfh cloud_liquid_ice: icmr cloud_liquid_water: clwmr + snow_water: snmr + rain_water: rwmr + graupel: grle ozone_mass_mixing_ratio: o3mr air_pressure_thickness: dpres layer_thickness: delz @@ -55,6 +61,9 @@ increment: - water_vapor_mixing_ratio_wrt_moist_air - cloud_liquid_ice - cloud_liquid_water + - snow_water + - rain_water + - graupel - ozone_mass_mixing_ratio - air_pressure_thickness - layer_thickness @@ -65,6 +74,9 @@ increment: water_vapor_mixing_ratio_wrt_moist_air: sphum_inc cloud_liquid_ice: icmr_inc cloud_liquid_water: liq_wat_inc + snow_water: snmr_inc + rain_water: rwmr_inc + graupel: grle_inc ozone_mass_mixing_ratio: o3mr_inc air_pressure_thickness: delp_inc layer_thickness: delz_inc @@ -83,6 +95,9 @@ output: water_vapor_mixing_ratio_wrt_moist_air: spfh cloud_liquid_ice: icmr cloud_liquid_water: clwmr + snow_water: snmr + rain_water: rwmr + graupel: grle ozone_mass_mixing_ratio: o3mr air_pressure_thickness: dpres layer_thickness: delz diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_correction_increment.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_correction_increment.yaml.j2 index 1f996bacc..170392af0 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_correction_increment.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_correction_increment.yaml.j2 @@ -3,6 +3,9 @@ increment variables: - layer_thickness - air_pressure_thickness - cloud_liquid_ice +- snow_water +- rain_water +- graupel - ozone_mass_mixing_ratio - water_vapor_mixing_ratio_wrt_moist_air - air_temperature @@ -68,6 +71,9 @@ forecast hours: water_vapor_mixing_ratio_wrt_moist_air: spfh cloud_liquid_ice: icmr cloud_liquid_water: clwmr + snow_water: snmr + rain_water: rwmr + graupel: grle ozone_mass_mixing_ratio: o3mr air_pressure_thickness: dpres layer_thickness: delz @@ -83,6 +89,9 @@ forecast hours: water_vapor_mixing_ratio_wrt_moist_air: sphum_inc cloud_liquid_ice: icmr_inc cloud_liquid_water: liq_wat_inc + snow_water: snmr_inc + rain_water: rwmr_inc + graupel: grle_inc ozone_mass_mixing_ratio: o3mr_inc air_pressure_thickness: delp_inc layer_thickness: delz_inc @@ -103,6 +112,9 @@ forecast hours: water_vapor_mixing_ratio_wrt_moist_air: sphum_inc cloud_liquid_ice: icmr_inc cloud_liquid_water: liq_wat_inc + snow_water: snmr_inc + rain_water: rwmr_inc + graupel: grle_inc ozone_mass_mixing_ratio: o3mr_inc air_pressure_thickness: delp_inc layer_thickness: delz_inc diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_ensemble_recenter.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_ensemble_recenter.yaml.j2 index f68e2e106..f71b855ad 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_ensemble_recenter.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_ensemble_recenter.yaml.j2 @@ -32,6 +32,9 @@ members: - water_vapor_mixing_ratio_wrt_moist_air - cloud_liquid_ice - cloud_liquid_water + - snow_water + - rain_water + - graupel - ozone_mass_mixing_ratio - air_pressure_thickness - layer_thickness @@ -42,6 +45,9 @@ members: water_vapor_mixing_ratio_wrt_moist_air: sphum_inc cloud_liquid_ice: icmr_inc cloud_liquid_water: liq_wat_inc + snow_water: snmr_inc + rain_water: rwmr_inc + graupel: grle_inc ozone_mass_mixing_ratio: o3mr_inc air_pressure_thickness: delp_inc layer_thickness: delz_inc @@ -58,6 +64,9 @@ members: - water_vapor_mixing_ratio_wrt_moist_air - cloud_liquid_ice - cloud_liquid_water + - snow_water + - rain_water + - graupel - ozone_mass_mixing_ratio - air_pressure_thickness - layer_thickness @@ -68,6 +77,9 @@ members: water_vapor_mixing_ratio_wrt_moist_air: sphum_inc cloud_liquid_ice: icmr_inc cloud_liquid_water: liq_wat_inc + snow_water: snmr_inc + rain_water: rwmr_inc + graupel: grle_inc ozone_mass_mixing_ratio: o3mr_inc air_pressure_thickness: delp_inc layer_thickness: delz_inc @@ -87,6 +99,9 @@ members: water_vapor_mixing_ratio_wrt_moist_air: sphum_inc cloud_liquid_ice: icmr_inc cloud_liquid_water: liq_wat_inc + snow_water: snmr_inc + rain_water: rwmr_inc + graupel: grle_inc ozone_mass_mixing_ratio: o3mr_inc air_pressure_thickness: delp_inc layer_thickness: delz_inc diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 index ce77c7ed4..931572742 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 @@ -8,6 +8,9 @@ variable change: - water_vapor_mixing_ratio_wrt_moist_air - cloud_liquid_ice - cloud_liquid_water + - snow_water + - rain_water + - graupel - ozone_mass_mixing_ratio - geopotential_height_at_surface output variables: &fv3incrvars @@ -17,6 +20,9 @@ variable change: - water_vapor_mixing_ratio_wrt_moist_air - cloud_liquid_ice - cloud_liquid_water + - snow_water + - rain_water + - graupel - ozone_mass_mixing_ratio - air_pressure_thickness - layer_thickness @@ -28,6 +34,9 @@ jedi increment variables: - water_vapor_mixing_ratio_wrt_moist_air - cloud_liquid_ice - cloud_liquid_water +- snow_water +- rain_water +- graupel - ozone_mass_mixing_ratio fv3 increment variables: *fv3incrvars background geometry: @@ -85,6 +94,9 @@ members from template: water_vapor_mixing_ratio_wrt_moist_air: spfh cloud_liquid_ice: icmr cloud_liquid_water: clwmr + snow_water: snmr + rain_water: rwmr + graupel: grle ozone_mass_mixing_ratio: o3mr geopotential_height_at_surface: hgtsfc jedi increment input: @@ -106,6 +118,9 @@ members from template: cloud_liquid_water: liq_wat_inc ozone_mass_mixing_ratio: o3mr_inc cloud_liquid_ice: icmr_inc + snow_water: snmr_inc + rain_water: rwmr_inc + graupel: grle_inc air_pressure_thickness: delp_inc layer_thickness: delz_inc pattern: '%mem%' diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 index 97e44406b..40da72655 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_variational.yaml.j2 @@ -9,6 +9,9 @@ variable change: - cloud_liquid_ice - cloud_liquid_water - ozone_mass_mixing_ratio + - snow_water + - rain_water + - graupel - geopotential_height_at_surface output variables: &fv3incrvars - eastward_wind @@ -17,6 +20,9 @@ variable change: - water_vapor_mixing_ratio_wrt_moist_air - cloud_liquid_ice - cloud_liquid_water + - snow_water + - rain_water + - graupel - ozone_mass_mixing_ratio - air_pressure_thickness - layer_thickness @@ -28,6 +34,9 @@ jedi increment variables: - water_vapor_mixing_ratio_wrt_moist_air - cloud_liquid_ice - cloud_liquid_water +- snow_water +- rain_water +- graupel - ozone_mass_mixing_ratio fv3 increment variables: *fv3incrvars background geometry: @@ -84,6 +93,9 @@ members: water_vapor_mixing_ratio_wrt_moist_air: spfh cloud_liquid_ice: icmr cloud_liquid_water: clwmr + snow_water: snmr + rain_water: rwmr + graupel: grle ozone_mass_mixing_ratio: o3mr geopotential_height_at_surface: hgtsfc jedi increment input: @@ -111,6 +123,9 @@ members: cloud_liquid_water: liq_wat_inc ozone_mass_mixing_ratio: o3mr_inc cloud_liquid_ice: icmr_inc + snow_water: snmr_inc + rain_water: rwmr_inc + graupel: grle_inc air_pressure_thickness: delp_inc layer_thickness: delz_inc # Optionally test the application diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 index 80b3f489e..316dc6d6e 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_background_error_hybrid_gsibec_bump.yaml.j2 @@ -29,7 +29,7 @@ components: local interpolator type: oops unstructured grid interpolator state variables to inverse: [eastward_wind,northward_wind,air_temperature,air_pressure_at_surface, water_vapor_mixing_ratio_wrt_moist_air,cloud_liquid_ice,cloud_liquid_water, - ozone_mass_mixing_ratio] + snow_water,rain_water,graupel,ozone_mass_mixing_ratio] linear variable change: linear variable change name: Control2Analysis input variables: @@ -40,6 +40,9 @@ components: - water_vapor_mixing_ratio_wrt_moist_air - cloud_liquid_ice - cloud_liquid_water + - snow_water + - rain_water + - graupel - mole_fraction_of_ozone_in_air output variables: - eastward_wind @@ -49,6 +52,9 @@ components: - water_vapor_mixing_ratio_wrt_moist_air - cloud_liquid_ice - cloud_liquid_water + - snow_water + - rain_water + - graupel - ozone_mass_mixing_ratio weight: value: 0.125 @@ -67,6 +73,9 @@ components: - water_vapor_mixing_ratio_wrt_moist_air - cloud_liquid_ice - cloud_liquid_water + - snow_water + - rain_water + - graupel - ozone_mass_mixing_ratio field io names: eastward_wind: ugrd @@ -76,6 +85,9 @@ components: water_vapor_mixing_ratio_wrt_moist_air: spfh cloud_liquid_ice: icmr cloud_liquid_water: clwmr + snow_water: snmr + rain_water: rwmr + graupel: grle ozone_mass_mixing_ratio: o3mr datapath: {{ atmosphere_background_ensemble_path }} filename is datetime templated: true @@ -98,6 +110,9 @@ components: - water_vapor_mixing_ratio_wrt_moist_air - cloud_liquid_ice - cloud_liquid_water + - snow_water + - rain_water + - graupel - ozone_mass_mixing_ratio read: general: @@ -255,6 +270,9 @@ components: - water_vapor_mixing_ratio_wrt_moist_air - cloud_liquid_ice - cloud_liquid_water + - snow_water + - rain_water + - graupel - ozone_mass_mixing_ratio output variables: - eastward_wind @@ -264,6 +282,9 @@ components: - water_vapor_mixing_ratio_wrt_moist_air - cloud_liquid_ice - cloud_liquid_water + - snow_water + - rain_water + - graupel - ozone_mass_mixing_ratio weight: value: 0.875 diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_cubed_sphere.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_cubed_sphere.yaml.j2 index 0f902d716..a3af136c4 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_cubed_sphere.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_final_increment_cubed_sphere.yaml.j2 @@ -11,6 +11,9 @@ output: - water_vapor_mixing_ratio_wrt_moist_air - cloud_liquid_ice - cloud_liquid_water + - snow_water + - rain_water + - graupel - ozone_mass_mixing_ratio field io names: eastward_wind: ugrd @@ -20,6 +23,9 @@ output: water_vapor_mixing_ratio_wrt_moist_air: spfh cloud_liquid_ice: icmr cloud_liquid_water: clwmr + snow_water: snmr + rain_water: rwmr + graupel: grle ozone_mass_mixing_ratio: o3mr geometry: fms initialization: diff --git a/parm/jcb-gdas/model/atmosphere/atmosphere_output_ensemble_increments_cubed_sphere.yaml.j2 b/parm/jcb-gdas/model/atmosphere/atmosphere_output_ensemble_increments_cubed_sphere.yaml.j2 index b835948f5..7f159e380 100644 --- a/parm/jcb-gdas/model/atmosphere/atmosphere_output_ensemble_increments_cubed_sphere.yaml.j2 +++ b/parm/jcb-gdas/model/atmosphere/atmosphere_output_ensemble_increments_cubed_sphere.yaml.j2 @@ -9,4 +9,7 @@ field io names: water_vapor_mixing_ratio_wrt_moist_air: spfh cloud_liquid_ice: icmr cloud_liquid_water: clwmr + snow_water: snmr + rain_water: rwmr + graupel: grle ozone_mass_mixing_ratio: o3mr diff --git a/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 index 5812f0b04..6b8492d15 100644 --- a/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 @@ -33,6 +33,8 @@ Sensor_ID: &{{observation_from_jcb}}_sensor_id atms_n20 EndianType: little_endian CoefficientPath: "{{crtm_coefficient_path}}" + do fov average: true + fov sample points per semimajor axis: 4 linear obs operator: Absorbers: [H2O, O3] From eba5e9098a423eb5804ac628704441ae9fc6809a Mon Sep 17 00:00:00 2001 From: "Tseganeh Z. Gichamo" Date: Thu, 18 Dec 2025 09:46:01 -0500 Subject: [PATCH 182/199] Add smap soil moisture config files (#218) Adds config files for SMAP soil moisture data type. The changes here are intended to be used for JEDI soil DA in the land offline workflow (outside of GW). The corresponding pygfs changes are included in the PR https://github.com/NOAA-PSL/land-offline_workflow/pull/58 Companion GDASApp PR: https://github.com/NOAA-EMC/GDASApp/pull/2011 --------- Co-authored-by: Jiarui Dong --- .../algorithm/soil/soil_addincrement.yaml.j2 | 74 +++++++ .../soil/soil_3dvar_outer_loop_1.yaml.j2 | 29 +++ .../model/soil/soil_background.yaml.j2 | 23 ++ .../model/soil/soil_background_error.yaml.j2 | 40 ++++ .../soil/soil_final_increment_fms.yaml.j2 | 28 +++ .../soil/soil_geometry_background.yaml.j2 | 10 + parm/jcb-gdas/model/soil/soil_output.yaml.j2 | 23 ++ .../snow/t00z/ghcn_snow.yaml | 2 +- .../snow/t06z/ghcn_snow.yaml | 2 +- .../observation_chronicle/soil/smap_soil.yaml | 14 ++ .../observations/soil/smap_soil.yaml.j2 | 196 ++++++++++++++++++ 11 files changed, 439 insertions(+), 2 deletions(-) create mode 100644 parm/jcb-gdas/algorithm/soil/soil_addincrement.yaml.j2 create mode 100644 parm/jcb-gdas/model/soil/soil_3dvar_outer_loop_1.yaml.j2 create mode 100644 parm/jcb-gdas/model/soil/soil_background.yaml.j2 create mode 100644 parm/jcb-gdas/model/soil/soil_background_error.yaml.j2 create mode 100644 parm/jcb-gdas/model/soil/soil_final_increment_fms.yaml.j2 create mode 100644 parm/jcb-gdas/model/soil/soil_geometry_background.yaml.j2 create mode 100644 parm/jcb-gdas/model/soil/soil_output.yaml.j2 create mode 100644 parm/jcb-gdas/observation_chronicle/soil/smap_soil.yaml create mode 100644 parm/jcb-gdas/observations/soil/smap_soil.yaml.j2 diff --git a/parm/jcb-gdas/algorithm/soil/soil_addincrement.yaml.j2 b/parm/jcb-gdas/algorithm/soil/soil_addincrement.yaml.j2 new file mode 100644 index 000000000..d6f01865f --- /dev/null +++ b/parm/jcb-gdas/algorithm/soil/soil_addincrement.yaml.j2 @@ -0,0 +1,74 @@ +state geometry: + fms initialization: + namelist filename: ./fv3jedi/fmsmpp.nml + field table filename: ./fv3jedi/field_table + akbk: ./fv3jedi/akbk.nc4 + layout: + - {{ soil_layout_x }} + - {{ soil_layout_y }} + npx: {{ soil_npx_ges }} + npy: {{ soil_npy_ges }} + npz: {{ soil_npz_ges }} +# field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_history.yaml +increment geometry: + fms initialization: + namelist filename: ./fv3jedi/fmsmpp.nml + field table filename: ./fv3jedi/field_table + akbk: ./fv3jedi/akbk.nc4 + layout: + - {{ soil_layout_x }} + - {{ soil_layout_y }} + npx: {{ soil_npx_anl }} + npy: {{ soil_npy_anl }} + npz: {{ soil_npz_anl }} +# field metadata override: ./fv3jedi/fv3jedi_fieldmetadata_fv3inc.yaml +state: + state variables: + - soilMoistureVolumetric + - stc + - totalSnowDepth + - sheleg + - vtype + - slmsk + - fraction_of_ice + field io names: + totalSnowDepth: snodl + soilMoistureVolumetric: smc + fraction_of_ice: fice + datapath: ./bkg + filetype: fms restart + skip coupler file: true + datetime: '{{ soil_background_time_iso }}' + filename_sfcd: {{ soil_background_time_fv3 }}.sfc_data.nc +increment: + added variables: + - soilMoistureVolumetric + field io names: + soilMoistureVolumetric: smc + datapath: ./anl + filetype: fms restart + skip coupler file: true + prefix: soilinc + filename is datetime templated: true + datetime: '{{ soil_background_time_iso }}' + filename_sfcd: {{ soil_background_time_fv3 }}.sfc_data.nc +output: + datapath: ./anl + prefix: soilanl + filetype: fms restart + skip coupler file: true + filename is datetime templated: true + datetime: '{{ soil_background_time_iso }}' + filename_sfcd: {{ soil_background_time_fv3 }}.sfc_data.nc + state variables: + - soilMoistureVolumetric + - stc + - totalSnowDepth + - sheleg + - vtype + - slmsk + - fraction_of_ice + field io names: + soilMoistureVolumetric: smc + totalSnowDepth: snodl + fraction_of_ice: fice diff --git a/parm/jcb-gdas/model/soil/soil_3dvar_outer_loop_1.yaml.j2 b/parm/jcb-gdas/model/soil/soil_3dvar_outer_loop_1.yaml.j2 new file mode 100644 index 000000000..302fd4a33 --- /dev/null +++ b/parm/jcb-gdas/model/soil/soil_3dvar_outer_loop_1.yaml.j2 @@ -0,0 +1,29 @@ +- ninner: 50 + gradient norm reduction: 1e-10 + test: on + geometry: + fms initialization: + namelist filename: {{ soil_fv3jedi_files_path }}/fmsmpp.nml + field table filename: {{ soil_fv3jedi_files_path }}/field_table + akbk: {{ soil_fv3jedi_files_path }}/akbk.nc4 + layout: + - {{ soil_layout_x }} + - {{ soil_layout_y }} + npx: {{ soil_npx_anl }} + npy: {{ soil_npy_anl }} + npz: {{ soil_npz_anl }} + time invariant fields: + state fields: + datetime: '{{ soil_background_time_iso }}' + filetype: fms restart + skip coupler file: true + state variables: + - filtered_orography + - fraction_of_land + field io names: + filtered_orography: orog_filt + fraction_of_land: land_frac + datapath: {{ soil_orog_files_path }}/ + filename_orog: {{ soil_orog_prefix }}_oro_data.nc + diagnostics: + departures: bkgmob diff --git a/parm/jcb-gdas/model/soil/soil_background.yaml.j2 b/parm/jcb-gdas/model/soil/soil_background.yaml.j2 new file mode 100644 index 000000000..c495e93f9 --- /dev/null +++ b/parm/jcb-gdas/model/soil/soil_background.yaml.j2 @@ -0,0 +1,23 @@ +datapath: {{ soil_background_path }} +filetype: fms restart +skip coupler file: true +datetime: '{{ soil_background_time_iso }}' +state variables: +- soilMoistureVolumetric +- stc +- totalSnowDepth +- sheleg +- vtype +- slmsk +- filtered_orography +- fraction_of_ice +- fraction_of_land +field io names: + soilMoistureVolumetric: smc + totalSnowDepth: snodl + filtered_orography: orog_filt + fraction_of_ice: fice + fraction_of_land: land_frac +filename_sfcd: '{{ soil_background_time_fv3 }}.sfc_data.nc' +filename_cplr: '{{ soil_background_time_fv3 }}.coupler.res' +filename_orog: '{{ soil_orog_prefix }}_oro_data.nc' diff --git a/parm/jcb-gdas/model/soil/soil_background_error.yaml.j2 b/parm/jcb-gdas/model/soil/soil_background_error.yaml.j2 new file mode 100644 index 000000000..c1220bea3 --- /dev/null +++ b/parm/jcb-gdas/model/soil/soil_background_error.yaml.j2 @@ -0,0 +1,40 @@ +covariance model: SABER +saber central block: + saber block name: BUMP_NICAS + calibration: #read: + general: + universe length-scale: 5000000.0 + drivers: + multivariate strategy: univariate + compute nicas: true #read global nicas: true + write global nicas: true + nicas: + resolution: 4 + explicit length-scales: true + horizontal length-scale: + - groups: + - soilMoistureVolumetric + value: 1000000.0 + vertical length-scale: + - groups: + - soilMoistureVolumetric + value: 1500.0 + interpolation type: + - groups: + - soilMoistureVolumetric + type: c0 + same horizontal convolution: true + io: + data directory: ./berror + files prefix: smc_bump_nicas_1000km_{{ soil_background_time_fv3 }} +saber outer blocks: +- saber block name: BUMP_StdDev + read: + drivers: + compute variance: true + variance: + explicit stddev: true + stddev: + - variables: + - soilMoistureVolumetric + value: 0.2 diff --git a/parm/jcb-gdas/model/soil/soil_final_increment_fms.yaml.j2 b/parm/jcb-gdas/model/soil/soil_final_increment_fms.yaml.j2 new file mode 100644 index 000000000..b3a7f22f8 --- /dev/null +++ b/parm/jcb-gdas/model/soil/soil_final_increment_fms.yaml.j2 @@ -0,0 +1,28 @@ +output: + state component: + datapath: ./anl + prefix: soilinc + filetype: fms restart + filename_sfcd: '{{ soil_background_time_fv3 }}.sfc_data.nc' + filename_cplr: '{{ soil_background_time_fv3 }}.coupler.res' + state variables: + - soilMoistureVolumetric + - stc + - totalSnowDepth + - sheleg + - vtype + - slmsk + field io names: + soilMoistureVolumetric: smc + totalSnowDepth: snodl +geometry: + fms initialization: + namelist filename: "{{ soil_fv3jedi_files_path }}/fmsmpp.nml" + field table filename: "{{ soil_fv3jedi_files_path }}/field_table" + akbk: "{{ soil_fv3jedi_files_path }}/akbk.nc4" + layout: + - {{ soil_layout_x }} + - {{ soil_layout_y }} + npx: {{ soil_npx_ges }} + npy: {{ soil_npy_ges }} + npz: {{ soil_npz_ges }} diff --git a/parm/jcb-gdas/model/soil/soil_geometry_background.yaml.j2 b/parm/jcb-gdas/model/soil/soil_geometry_background.yaml.j2 new file mode 100644 index 000000000..d60dc8a36 --- /dev/null +++ b/parm/jcb-gdas/model/soil/soil_geometry_background.yaml.j2 @@ -0,0 +1,10 @@ +fms initialization: + namelist filename: "{{soil_fv3jedi_files_path}}/fmsmpp.nml" + field table filename: "{{soil_fv3jedi_files_path}}/field_table" +akbk: "{{soil_fv3jedi_files_path}}/akbk.nc4" +layout: +- {{soil_layout_x}} +- {{soil_layout_y}} +npx: {{soil_npx_ges}} +npy: {{soil_npy_ges}} +npz: {{soil_npz_ges}} diff --git a/parm/jcb-gdas/model/soil/soil_output.yaml.j2 b/parm/jcb-gdas/model/soil/soil_output.yaml.j2 new file mode 100644 index 000000000..83e44b6ee --- /dev/null +++ b/parm/jcb-gdas/model/soil/soil_output.yaml.j2 @@ -0,0 +1,23 @@ +state variables: +- soilMoistureVolumetric +- stc +- totalSnowDepth +- sheleg +- vtype +- slmsk +- filtered_orography +- fraction_of_ice +- fraction_of_land +field io names: + soilMoistureVolumetric: smc + totalSnowDepth: snodl + filtered_orography: orog_filt + fraction_of_ice: fice + fraction_of_land: land_frac +datapath: ./anl +prefix: soil_finanl +filetype: fms restart +datetime: '{{ soil_background_time_iso }}' +filename_sfcd: '{{ soil_background_time_fv3 }}.sfc_data.nc' +filename_cplr: '{{ soil_background_time_fv3 }}.coupler.res' +filename_orog: '{{ soil_orog_prefix }}_oro_data.nc' diff --git a/parm/jcb-gdas/observation_chronicle/snow/t00z/ghcn_snow.yaml b/parm/jcb-gdas/observation_chronicle/snow/t00z/ghcn_snow.yaml index f7db866a8..20420eabd 100644 --- a/parm/jcb-gdas/observation_chronicle/snow/t00z/ghcn_snow.yaml +++ b/parm/jcb-gdas/observation_chronicle/snow/t00z/ghcn_snow.yaml @@ -1,6 +1,6 @@ # Instrument metadata # ------------------- -commissioned: 1970-01-01T00:00:00 +commissioned: 1900-01-01T00:00:00 observer_type: conventional # Type of chronicle to use diff --git a/parm/jcb-gdas/observation_chronicle/snow/t06z/ghcn_snow.yaml b/parm/jcb-gdas/observation_chronicle/snow/t06z/ghcn_snow.yaml index f7db866a8..20420eabd 100644 --- a/parm/jcb-gdas/observation_chronicle/snow/t06z/ghcn_snow.yaml +++ b/parm/jcb-gdas/observation_chronicle/snow/t06z/ghcn_snow.yaml @@ -1,6 +1,6 @@ # Instrument metadata # ------------------- -commissioned: 1970-01-01T00:00:00 +commissioned: 1900-01-01T00:00:00 observer_type: conventional # Type of chronicle to use diff --git a/parm/jcb-gdas/observation_chronicle/soil/smap_soil.yaml b/parm/jcb-gdas/observation_chronicle/soil/smap_soil.yaml new file mode 100644 index 000000000..af70c2a9d --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/soil/smap_soil.yaml @@ -0,0 +1,14 @@ +# Instrument metadata +# ------------------- +commissioned: 2015-01-31T00:00:00 + +observer_type: conventional # Type of chronicle to use + +window_option: max + +# observation type initial configuration +# -------------------------------- +stations_to_reject: [fakelist] + +# Chronicle of changes for this observation type +# ---------------------------------------- diff --git a/parm/jcb-gdas/observations/soil/smap_soil.yaml.j2 b/parm/jcb-gdas/observations/soil/smap_soil.yaml.j2 new file mode 100644 index 000000000..ff7e9a575 --- /dev/null +++ b/parm/jcb-gdas/observations/soil/smap_soil.yaml.j2 @@ -0,0 +1,196 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: smap_soil + obsdatain: + engine: + type: H5File + obsfile: "{{soil_obsdatain_path}}/{{soil_obsdatain_prefix}}{{observation_from_jcb}}.nc" + missing file action: warn + obsdataout: + engine: + type: H5File + obsfile: "{{soil_obsdataout_path}}/{{soil_obsdataout_prefix}}{{observation_from_jcb}}{{soil_obsdataout_suffix}}" + simulated variables: [soilMoistureVolumetric] + # + + # Observation Operator + # -------------------- + obs operator: + name: Identity + # + + # Observation Filters (QC) + # ------------------------ + obs pre filters: + - filter: Create Diagnostic Flags + flags: + - name: missing_smc + initial value: false +# - name: missing_elevation +# initial value: false + - name: invalid_smc + initial value: false +# - name: invalid_elevation +# initial value: false + - name: land_check + initial value: false + - name: landice_check + initial value: false + - name: seaice_check + initial value: false + - name: snow_check + initial value: false + - name: retrv_qual_check + initial value: false + - name: vegtype_check + initial value: false + - name: elevation_bkgdiff + initial value: false +# - name: rejectlist +# initial value: false + - name: background_check + initial value: false + - filter: Perform Action + filter variables: + - name: soilMoistureVolumetric + action: + name: assign error + error parameter: 0.20 + - filter: Domain Check + where: + - variable: + name: ObsValue/soilMoistureVolumetric + value: is_valid + actions: + - name: set + flag: missing_smc + ignore: rejected observations + - name: reject +# - filter: Domain Check +# where: +# - variable: +# name: MetaData/stationElevation +# value: is_valid +# actions: +# - name: set +# flag: missing_elevation +# ignore: rejected observations +# - name: reject + obs prior filters: + - filter: Bounds Check + filter variables: + - name: soilMoistureVolumetric + minvalue: 0.0 + maxvalue: 1.0 + actions: + - name: set + flag: invalid_smc + ignore: rejected observations + - name: reject + - filter: Domain Check # land only + where: + - variable: + name: GeoVaLs/fraction_of_land + minvalue: 0.5 + actions: + - name: set + flag: land_check + ignore: rejected observations + - name: reject +# - filter: Domain Check +# where: +# - variable: +# name: MetaData/stationElevation +# minvalue: -200.0 +# maxvalue: 9000.0 +# actions: +# - name: set +# flag: invalid_elevation +# ignore: rejected observations +# - name: reject + - filter: Domain Check # land only, no sea ice + where: + - variable: + name: GeoVaLs/fraction_of_ice + maxvalue: 0.0 + actions: + - name: set + flag: seaice_check + ignore: rejected observations + - name: reject + - filter: Domain Check # no snow + where: + - variable: + name: GeoVaLs/sheleg + maxvalue: 0.0 + actions: + - name: set + flag: snow_check + ignore: rejected observations + - name: reject + - filter: Domain Check # high-quality retrievals + where: + - variable: + name: PreQC/soilMoistureVolumetric + is_in: 0, 8 + actions: + - name: set + flag: retrv_qual_check + ignore: rejected observations + - name: reject + - filter: RejectList # no land-ice + where: + - variable: + name: GeoVaLs/vtype + minvalue: 14.5 + maxvalue: 15.5 + actions: + - name: set + flag: landice_check + ignore: rejected observations + - name: reject + - filter: RejectList # exclude certain IGBP vegetation types + where: + - variable: + name: GeoVaLs/vtype + absolute_tolerance: 1.0e-3 + is_close_to_any_of: [1, 2, 3, 4, 13, 15, 17] + actions: + - name: set + flag: vegtype_check + ignore: rejected observations + - name: reject +# - filter: Difference Check # elevation check +# reference: MetaData/stationElevation +# value: GeoVaLs/filtered_orography +# threshold: 800. +# actions: +# - name: set +# flag: elevation_bkgdiff +# ignore: rejected observations +# - name: reject +# - filter: BlackList +# where: +# - variable: +# name: MetaData/stationIdentification +# is_in: {{ get_conventional_rejected_stations(observation_from_jcb) }} +# actions: +# - name: set +# flag: rejectlist +# ignore: rejected observations +# - name: reject + obs post filters: + - filter: Background Check # gross error check + filter variables: + - name: soilMoistureVolumetric + #threshold: 5 + absolute threshold: 0.5 + bias correction parameter: 1.0 + actions: + - name: set + flag: background_check + ignore: rejected observations + - name: reject From 2fcc3576a2c88d8f98b9f5e0ad8a5aa40c1aae40 Mon Sep 17 00:00:00 2001 From: DavidNew-NOAA <134300700+DavidNew-NOAA@users.noreply.github.com> Date: Fri, 19 Dec 2025 10:57:15 -0500 Subject: [PATCH 183/199] JCB-GDAS companion to GW obs staging and analysis stats refactor (#214) This PR makes changes which support changes made in https://github.com/NOAA-EMC/global-workflow/pull/4306 and https://github.com/NOAA-EMC/GDASApp/pull/1999 in the following ways: 1. The obs statistics templates are moved from `algorithms/obsstats` to `model/obsstats`, since these are indeed model templates and not standalone algorithms. 2. The algorithm YAML for the JEDI obs stats application is moved from the jcb-algorithms repo to this repo in `algorithms/obsstats`. The justification is that the algorithm templates in jcb-algorithms should be for general model-agnostics JEDI applications, usually existing in the OOPS repo. This application on the other hand is specific to GDASApp. 3. Minor variable names changes are made in the obs statistics templates. --- .../algorithm/observation_statistics/anlstat.yaml.j2 | 11 +++++++++++ .../aero/retrieval_aod_viirs_n20.yaml.j2} | 5 +++-- .../aero/retrieval_aod_viirs_n21.yaml.j2} | 5 +++-- .../aero/retrieval_aod_viirs_npp.yaml.j2} | 5 +++-- .../atmosphere/conventional_ps.yaml.j2} | 7 ++++--- .../atmosphere/prepbufr_adpsfc.yaml.j2} | 5 +++-- .../atmosphere/prepbufr_adpupa.yaml.j2} | 5 +++-- .../atmosphere/prepbufr_sfcshp.yaml.j2} | 5 +++-- .../atmosphere_gsi/gsi_sfc_ps.yaml.j2} | 5 +++-- .../snow/ims_snow.yaml.j2} | 5 +++-- .../snow/madis_snow.yaml.j2} | 5 +++-- .../snow/sfcsno.yaml.j2} | 5 +++-- .../snow/snocvr.yaml.j2} | 5 +++-- .../snow/snocvr_snomad.yaml.j2} | 5 +++-- .../snow/snomad.yaml.j2} | 5 +++-- 15 files changed, 54 insertions(+), 29 deletions(-) create mode 100644 parm/jcb-gdas/algorithm/observation_statistics/anlstat.yaml.j2 rename parm/jcb-gdas/{algorithm/obstats/aero/retrieval_aod_viirs_n20_template.yaml.j2 => observation_statistics/aero/retrieval_aod_viirs_n20.yaml.j2} (79%) rename parm/jcb-gdas/{algorithm/obstats/aero/retrieval_aod_viirs_n21_template.yaml.j2 => observation_statistics/aero/retrieval_aod_viirs_n21.yaml.j2} (79%) rename parm/jcb-gdas/{algorithm/obstats/aero/retrieval_aod_viirs_npp_template.yaml.j2 => observation_statistics/aero/retrieval_aod_viirs_npp.yaml.j2} (79%) rename parm/jcb-gdas/{algorithm/obstats/atmos/conventional_ps_template.yaml.j2 => observation_statistics/atmosphere/conventional_ps.yaml.j2} (75%) rename parm/jcb-gdas/{algorithm/obstats/atmos/prepbufr_adpsfc_template.yaml.j2 => observation_statistics/atmosphere/prepbufr_adpsfc.yaml.j2} (80%) rename parm/jcb-gdas/{algorithm/obstats/atmos/prepbufr_adpupa_template.yaml.j2 => observation_statistics/atmosphere/prepbufr_adpupa.yaml.j2} (80%) rename parm/jcb-gdas/{algorithm/obstats/atmos/prepbufr_sfcshp_template.yaml.j2 => observation_statistics/atmosphere/prepbufr_sfcshp.yaml.j2} (80%) rename parm/jcb-gdas/{algorithm/obstats/atmos_gsi/gsi_sfc_ps_template.yaml.j2 => observation_statistics/atmosphere_gsi/gsi_sfc_ps.yaml.j2} (79%) rename parm/jcb-gdas/{algorithm/obstats/snow/ims_snow_template.yaml.j2 => observation_statistics/snow/ims_snow.yaml.j2} (84%) rename parm/jcb-gdas/{algorithm/obstats/snow/madis_snow_template.yaml.j2 => observation_statistics/snow/madis_snow.yaml.j2} (84%) rename parm/jcb-gdas/{algorithm/obstats/snow/sfcsno_template.yaml.j2 => observation_statistics/snow/sfcsno.yaml.j2} (84%) rename parm/jcb-gdas/{algorithm/obstats/snow/snocvr_template.yaml.j2 => observation_statistics/snow/snocvr.yaml.j2} (84%) rename parm/jcb-gdas/{algorithm/obstats/snow/snocvr_snomad_template.yaml.j2 => observation_statistics/snow/snocvr_snomad.yaml.j2} (84%) rename parm/jcb-gdas/{algorithm/obstats/snow/snomad_template.yaml.j2 => observation_statistics/snow/snomad.yaml.j2} (84%) diff --git a/parm/jcb-gdas/algorithm/observation_statistics/anlstat.yaml.j2 b/parm/jcb-gdas/algorithm/observation_statistics/anlstat.yaml.j2 new file mode 100644 index 000000000..420544cce --- /dev/null +++ b/parm/jcb-gdas/algorithm/observation_statistics/anlstat.yaml.j2 @@ -0,0 +1,11 @@ +time window: + begin: '{{ current_cycle_iso }}' + length: '{{ window_length }}' + bound to include: '{{ bound_to_include | default("begin", true) }}' + +observers: +{% for observation_from_jcb in observations %} +{% filter indent(width=2) %} +{% include observation_from_jcb + '.yaml.j2' %} +{% endfilter %} +{% endfor %} diff --git a/parm/jcb-gdas/algorithm/obstats/aero/retrieval_aod_viirs_n20_template.yaml.j2 b/parm/jcb-gdas/observation_statistics/aero/retrieval_aod_viirs_n20.yaml.j2 similarity index 79% rename from parm/jcb-gdas/algorithm/obstats/aero/retrieval_aod_viirs_n20_template.yaml.j2 rename to parm/jcb-gdas/observation_statistics/aero/retrieval_aod_viirs_n20.yaml.j2 index 3bed75d11..efc7eeb68 100644 --- a/parm/jcb-gdas/algorithm/obstats/aero/retrieval_aod_viirs_n20_template.yaml.j2 +++ b/parm/jcb-gdas/observation_statistics/aero/retrieval_aod_viirs_n20.yaml.j2 @@ -3,14 +3,15 @@ obsdatain: engine: type: H5File - obsfile: {{ aero_obsdatain_path }}/aero/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc + obsfile: "{{ aero_obsdatain_path }}/{{ aero_obsdatain_prefix }}{{ observation_from_jcb }}{{ aero_obsdatain_suffix }}" simulated variables: ['aerosolOpticalDepth'] observed variables: ['aerosolOpticalDepth'] variables: ['aerosolOpticalDepth'] groups to process: ['ombg', 'oman'] qc groups: ['EffectiveQC0', 'EffectiveQC1'] statistics to compute: ['mean', 'count', 'RMS'] - output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_aod.nc" + output file: "{{ aero_obsdataout_path }}/{{ aero_obsdataout_prefix }}{{ observation_from_jcb }}{{ aero_obsdataout_suffix }}" + output ascii file: "{{ aero_obsdataout_path }}/{{ observation_from_jcb }}_ioda_stats.txt" regular grid binning: bin size in degrees: 5.0 use negative longitudes: true diff --git a/parm/jcb-gdas/algorithm/obstats/aero/retrieval_aod_viirs_n21_template.yaml.j2 b/parm/jcb-gdas/observation_statistics/aero/retrieval_aod_viirs_n21.yaml.j2 similarity index 79% rename from parm/jcb-gdas/algorithm/obstats/aero/retrieval_aod_viirs_n21_template.yaml.j2 rename to parm/jcb-gdas/observation_statistics/aero/retrieval_aod_viirs_n21.yaml.j2 index fc471d35e..b5ff3e457 100644 --- a/parm/jcb-gdas/algorithm/obstats/aero/retrieval_aod_viirs_n21_template.yaml.j2 +++ b/parm/jcb-gdas/observation_statistics/aero/retrieval_aod_viirs_n21.yaml.j2 @@ -3,14 +3,15 @@ obsdatain: engine: type: H5File - obsfile: {{ aero_obsdatain_path }}/aero/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc + obsfile: "{{ aero_obsdatain_path }}/{{ aero_obsdatain_prefix }}{{ observation_from_jcb }}{{ aero_obsdatain_suffix }}" simulated variables: ['aerosolOpticalDepth'] observed variables: ['aerosolOpticalDepth'] variables: ['aerosolOpticalDepth'] groups to process: ['ombg', 'oman'] qc groups: ['EffectiveQC0', 'EffectiveQC1'] statistics to compute: ['mean', 'count', 'RMS'] - output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_aod.nc" + output file: "{{ aero_obsdataout_path }}/{{ aero_obsdataout_prefix }}{{ observation_from_jcb }}{{ aero_obsdataout_suffix }}" + output ascii file: {{ aero_obsdataout_path }}/{{ observation_from_jcb }}_ioda_stats.txt regular grid binning: bin size in degrees: 5.0 use negative longitudes: true diff --git a/parm/jcb-gdas/algorithm/obstats/aero/retrieval_aod_viirs_npp_template.yaml.j2 b/parm/jcb-gdas/observation_statistics/aero/retrieval_aod_viirs_npp.yaml.j2 similarity index 79% rename from parm/jcb-gdas/algorithm/obstats/aero/retrieval_aod_viirs_npp_template.yaml.j2 rename to parm/jcb-gdas/observation_statistics/aero/retrieval_aod_viirs_npp.yaml.j2 index 31121e686..ddbe9adcf 100644 --- a/parm/jcb-gdas/algorithm/obstats/aero/retrieval_aod_viirs_npp_template.yaml.j2 +++ b/parm/jcb-gdas/observation_statistics/aero/retrieval_aod_viirs_npp.yaml.j2 @@ -3,14 +3,15 @@ obsdatain: engine: type: H5File - obsfile: {{ aero_obsdatain_path }}/aero/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc + obsfile: "{{ aero_obsdatain_path }}/{{ aero_obsdatain_prefix }}{{ observation_from_jcb }}{{ aero_obsdatain_suffix }}" simulated variables: ['aerosolOpticalDepth'] observed variables: ['aerosolOpticalDepth'] variables: ['aerosolOpticalDepth'] groups to process: ['ombg', 'oman'] qc groups: ['EffectiveQC0', 'EffectiveQC1'] statistics to compute: ['mean', 'count', 'RMS'] - output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_aod.nc" + output file: "{{ aero_obsdataout_path }}/{{ aero_obsdataout_prefix }}{{ observation_from_jcb }}{{ aero_obsdataout_suffix }}" + output ascii file: "{{ aero_obsdataout_path }}/{{ observation_from_jcb }}_ioda_stats.txt" regular grid binning: bin size in degrees: 5.0 use negative longitudes: true diff --git a/parm/jcb-gdas/algorithm/obstats/atmos/conventional_ps_template.yaml.j2 b/parm/jcb-gdas/observation_statistics/atmosphere/conventional_ps.yaml.j2 similarity index 75% rename from parm/jcb-gdas/algorithm/obstats/atmos/conventional_ps_template.yaml.j2 rename to parm/jcb-gdas/observation_statistics/atmosphere/conventional_ps.yaml.j2 index e38129ab3..704072d5e 100644 --- a/parm/jcb-gdas/algorithm/obstats/atmos/conventional_ps_template.yaml.j2 +++ b/parm/jcb-gdas/observation_statistics/atmosphere/conventional_ps.yaml.j2 @@ -3,14 +3,15 @@ obsdatain: engine: type: H5File - obsfile: {{ atmosphere_obsdatain_path }}/atmos/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc + obsfile: "{{ atmosphere_obsdatain_path }}/{{ atmosphere_obsdatain_prefix }}{{ observation_from_jcb }}{{ atmosphere_obsdatain_suffix }}" simulated variables: ['stationPressure'] observed variables: ['stationPressure'] statistics to compute: ['mean', 'count', 'RMS'] variables: ['stationPressure'] groups to process: ['ombg', 'oman'] qc groups: ['EffectiveQC0', 'EffectiveQC1'] - output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_atmos.nc" + output file: "{{ atmosphere_obsdataout_path }}/{{ atmosphere_obsdataout_prefix }}{{ observation_from_jcb }}{{ atmosphere_obsdataout_suffix }}" + output ascii file: "{{ atmosphere_obsdataout_path }}/{{ observation_from_jcb }}_ioda_stats.txt" regular grid binning: bin size in degrees: 1.0 use negative longitudes: true @@ -46,4 +47,4 @@ first mask variable: latitude first mask range: [0,70] second mask variable: longitude - second mask range: [38, 180] \ No newline at end of file + second mask range: [38, 180] diff --git a/parm/jcb-gdas/algorithm/obstats/atmos/prepbufr_adpsfc_template.yaml.j2 b/parm/jcb-gdas/observation_statistics/atmosphere/prepbufr_adpsfc.yaml.j2 similarity index 80% rename from parm/jcb-gdas/algorithm/obstats/atmos/prepbufr_adpsfc_template.yaml.j2 rename to parm/jcb-gdas/observation_statistics/atmosphere/prepbufr_adpsfc.yaml.j2 index 51aaca9de..44cfcb445 100644 --- a/parm/jcb-gdas/algorithm/obstats/atmos/prepbufr_adpsfc_template.yaml.j2 +++ b/parm/jcb-gdas/observation_statistics/atmosphere/prepbufr_adpsfc.yaml.j2 @@ -3,14 +3,15 @@ obsdatain: engine: type: H5File - obsfile: {{ atmosphere_obsdatain_path }}/atmos/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc + obsfile: "{{ atmosphere_obsdatain_path }}/{{ atmosphere_obsdatain_prefix }}{{ observation_from_jcb }}{{ atmosphere_obsdatain_suffix }}" simulated variables: ['stationPressure'] observed variables: ['stationPressure'] statistics to compute: ['mean', 'count', 'RMS'] variables: ['stationPressure'] groups to process: ['ombg', 'oman'] qc groups: ['EffectiveQC0', 'EffectiveQC1'] - output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_atmos.nc" + output file: "{{ atmosphere_obsdataout_path }}/{{ atmosphere_obsdataout_prefix }}{{ observation_from_jcb }}{{ atmosphere_obsdataout_suffix }}" + output ascii file: "{{ atmosphere_obsdataout_path }}/{{ observation_from_jcb }}_ioda_stats.txt" ascii vertical bins: vertical bin names: [1000+, 1000-900, 900-800, 800-600, 600-400, 400-300, 300-250, 250-200, 200-150, 150-100, 100-50] vertical bin ranges: [100000, 90000, 80000, 60000, 40000, 30000, 25000, 20000, 15000, 10000, 5000] diff --git a/parm/jcb-gdas/algorithm/obstats/atmos/prepbufr_adpupa_template.yaml.j2 b/parm/jcb-gdas/observation_statistics/atmosphere/prepbufr_adpupa.yaml.j2 similarity index 80% rename from parm/jcb-gdas/algorithm/obstats/atmos/prepbufr_adpupa_template.yaml.j2 rename to parm/jcb-gdas/observation_statistics/atmosphere/prepbufr_adpupa.yaml.j2 index a951d1e27..c732b701b 100644 --- a/parm/jcb-gdas/algorithm/obstats/atmos/prepbufr_adpupa_template.yaml.j2 +++ b/parm/jcb-gdas/observation_statistics/atmosphere/prepbufr_adpupa.yaml.j2 @@ -3,14 +3,15 @@ obsdatain: engine: type: H5File - obsfile: {{ atmosphere_obsdatain_path }}/atmos/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc + obsfile: "{{ atmosphere_obsdatain_path }}/{{ atmosphere_obsdatain_prefix }}{{ observation_from_jcb }}{{ atmosphere_obsdatain_suffix }}" simulated variables: ['stationPressure'] observed variables: ['stationPressure'] statistics to compute: ['mean', 'count', 'RMS'] variables: ['stationPressure'] groups to process: ['ombg', 'oman'] qc groups: ['EffectiveQC0', 'EffectiveQC1'] - output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_atmos.nc" + output file: "{{ atmosphere_obsdataout_path }}/{{ atmosphere_obsdataout_prefix }}{{ observation_from_jcb }}{{ atmosphere_obsdataout_suffix }}" + output ascii file: "{{ atmosphere_obsdataout_path }}/{{ observation_from_jcb }}_ioda_stats.txt" ascii vertical bins: vertical bin names: [1000+, 1000-900, 900-800, 800-600, 600-400, 400-300, 300-250, 250-200, 200-150, 150-100, 100-50] vertical bin ranges: [100000, 90000, 80000, 60000, 40000, 30000, 25000, 20000, 15000, 10000, 5000] diff --git a/parm/jcb-gdas/algorithm/obstats/atmos/prepbufr_sfcshp_template.yaml.j2 b/parm/jcb-gdas/observation_statistics/atmosphere/prepbufr_sfcshp.yaml.j2 similarity index 80% rename from parm/jcb-gdas/algorithm/obstats/atmos/prepbufr_sfcshp_template.yaml.j2 rename to parm/jcb-gdas/observation_statistics/atmosphere/prepbufr_sfcshp.yaml.j2 index 58539ad6f..9b24361c3 100644 --- a/parm/jcb-gdas/algorithm/obstats/atmos/prepbufr_sfcshp_template.yaml.j2 +++ b/parm/jcb-gdas/observation_statistics/atmosphere/prepbufr_sfcshp.yaml.j2 @@ -3,14 +3,15 @@ obsdatain: engine: type: H5File - obsfile: {{ atmosphere_obsdatain_path }}/atmos/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc + obsfile: "{{ atmosphere_obsdatain_path }}/{{ atmosphere_obsdatain_prefix }}{{ observation_from_jcb }}{{ atmosphere_obsdatain_suffix }}" simulated variables: ['stationPressure'] observed variables: ['stationPressure'] statistics to compute: ['mean', 'count', 'RMS'] variables: ['stationPressure'] groups to process: ['ombg', 'oman'] qc groups: ['EffectiveQC0', 'EffectiveQC1'] - output file: "{{ obspace }}_{{ stat_current_cycle_YMDH }}_output_atmos.nc" + output file: "{{ atmosphere_obsdataout_path }}/{{ atmosphere_obsdataout_prefix }}{{ observation_from_jcb }}{{ atmosphere_obsdataout_suffix }}" + output ascii file: "{{ atmosphere_obsdataout_path }}/{{ observation_from_jcb }}_ioda_stats.txt" ascii vertical bins: vertical bin names: [1000+, 1000-900, 900-800, 800-600, 600-400, 400-300, 300-250, 250-200, 200-150, 150-100, 100-50] vertical bin ranges: [100000, 90000, 80000, 60000, 40000, 30000, 25000, 20000, 15000, 10000, 5000] diff --git a/parm/jcb-gdas/algorithm/obstats/atmos_gsi/gsi_sfc_ps_template.yaml.j2 b/parm/jcb-gdas/observation_statistics/atmosphere_gsi/gsi_sfc_ps.yaml.j2 similarity index 79% rename from parm/jcb-gdas/algorithm/obstats/atmos_gsi/gsi_sfc_ps_template.yaml.j2 rename to parm/jcb-gdas/observation_statistics/atmosphere_gsi/gsi_sfc_ps.yaml.j2 index 568ee9aeb..88316c9e9 100644 --- a/parm/jcb-gdas/algorithm/obstats/atmos_gsi/gsi_sfc_ps_template.yaml.j2 +++ b/parm/jcb-gdas/observation_statistics/atmosphere_gsi/gsi_sfc_ps.yaml.j2 @@ -3,14 +3,15 @@ obsdatain: engine: type: H5File - obsfile: {{ atmosphere_obsdatain_path }}/atmos_gsi/sfc_ps_gsi_{{ stat_current_cycle_YMDH }}.nc4 + obsfile: "{{ atmosphere_gsi_obsdatain_path }}/{{ atmosphere_gsi_obsdatain_prefix }}{{ observation_from_jcb }}{{ atmosphere_gsi_obsdatain_suffix }}" simulated variables: ['stationPressure'] observed variables: ['stationPressure'] statistics to compute: ['mean', 'count', 'RMS'] variables: ['stationPressure'] groups to process: ['ombg', 'oman'] qc groups: ['GsiEffectiveQCGes', 'GsiEffectiveQCAnl'] - output file: "sfc_ps_{{ stat_current_cycle_YMDH }}_output_atmos_gsi.nc" + output file: "{{ atmosphere_gsi_obsdataout_path }}/{{ atmosphere_gsi_obsdataout_prefix }}{{ observation_from_jcb }}{{ atmosphere_gsi_obsdataout_suffix }}" + output ascii file: "{{ atmosphere_gsi_obsdataout_path }}/{{ observation_from_jcb }}_ioda_stats.txt" ascii vertical bins: vertical bin names: [1000+, 1000-900, 900-800, 800-600, 600-400, 400-300, 300-250, 250-200, 200-150, 150-100, 100-50] vertical bin ranges: [100000, 90000, 80000, 60000, 40000, 30000, 25000, 20000, 15000, 10000, 5000] diff --git a/parm/jcb-gdas/algorithm/obstats/snow/ims_snow_template.yaml.j2 b/parm/jcb-gdas/observation_statistics/snow/ims_snow.yaml.j2 similarity index 84% rename from parm/jcb-gdas/algorithm/obstats/snow/ims_snow_template.yaml.j2 rename to parm/jcb-gdas/observation_statistics/snow/ims_snow.yaml.j2 index 360bb48c1..7f584d785 100644 --- a/parm/jcb-gdas/algorithm/obstats/snow/ims_snow_template.yaml.j2 +++ b/parm/jcb-gdas/observation_statistics/snow/ims_snow.yaml.j2 @@ -3,14 +3,15 @@ obsdatain: engine: type: H5File - obsfile: {{ snow_obsdatain_path }}/snow/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc + obsfile: "{{ snow_obsdatain_path }}/{{ snow_obsdatain_prefix }}{{ observation_from_jcb }}{{ snow_obsdatain_suffix }}" simulated variables: ['totalSnowDepth'] observed variables: ['totalSnowDepth'] statistics to compute: ['mean', 'count', 'RMS'] variables: [totalSnowDepth] groups to process: ['ombg', 'oman'] qc groups: ['EffectiveQC0', 'EffectiveQC1'] - output file: "{{ stat_current_cycle_YMDH }}.ioda_hofx_stats.{{ obspace }}.nc" + output file: "{{ snow_obsdataout_path }}/{{ snow_obsdataout_prefix }}{{ observation_from_jcb }}{{ snow_obsdataout_suffix }}" + output ascii file: "{{ snow_obsdataout_path }}/{{ observation_from_jcb }}_ioda_stats.txt" regular grid binning: bin size in degrees: 2.0 use negative longitudes: false diff --git a/parm/jcb-gdas/algorithm/obstats/snow/madis_snow_template.yaml.j2 b/parm/jcb-gdas/observation_statistics/snow/madis_snow.yaml.j2 similarity index 84% rename from parm/jcb-gdas/algorithm/obstats/snow/madis_snow_template.yaml.j2 rename to parm/jcb-gdas/observation_statistics/snow/madis_snow.yaml.j2 index 9caabdb52..63fd6678b 100644 --- a/parm/jcb-gdas/algorithm/obstats/snow/madis_snow_template.yaml.j2 +++ b/parm/jcb-gdas/observation_statistics/snow/madis_snow.yaml.j2 @@ -3,14 +3,15 @@ obsdatain: engine: type: H5File - obsfile: {{ snow_obsdatain_path }}/snow/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc + obsfile: "{{ snow_obsdatain_path }}/{{ snow_obsdatain_prefix }}{{ observation_from_jcb }}{{ snow_obsdatain_suffix }}" simulated variables: ['totalSnowDepth'] observed variables: ['totalSnowDepth'] statistics to compute: ['mean', 'count', 'RMS'] variables: [totalSnowDepth] groups to process: ['ombg', 'oman'] qc groups: ['EffectiveQC0', 'EffectiveQC1'] - output file: "{{ stat_current_cycle_YMDH }}.ioda_hofx_stats.{{ obspace }}.nc" + output file: "{{ snow_obsdataout_path }}/{{ snow_obsdataout_prefix }}{{ observation_from_jcb }}{{ snow_obsdataout_suffix }}" + output ascii file: "{{ snow_obsdataout_path }}/{{ observation_from_jcb }}_ioda_stats.txt" regular grid binning: bin size in degrees: 1.0 use negative longitudes: true diff --git a/parm/jcb-gdas/algorithm/obstats/snow/sfcsno_template.yaml.j2 b/parm/jcb-gdas/observation_statistics/snow/sfcsno.yaml.j2 similarity index 84% rename from parm/jcb-gdas/algorithm/obstats/snow/sfcsno_template.yaml.j2 rename to parm/jcb-gdas/observation_statistics/snow/sfcsno.yaml.j2 index cc1c24648..818a23065 100644 --- a/parm/jcb-gdas/algorithm/obstats/snow/sfcsno_template.yaml.j2 +++ b/parm/jcb-gdas/observation_statistics/snow/sfcsno.yaml.j2 @@ -3,14 +3,15 @@ obsdatain: engine: type: H5File - obsfile: {{ snow_obsdatain_path }}/snow/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc + obsfile: "{{ snow_obsdatain_path }}/{{ snow_obsdatain_prefix }}{{ observation_from_jcb }}{{ snow_obsdatain_suffix }}" simulated variables: ['totalSnowDepth'] observed variables: ['totalSnowDepth'] statistics to compute: ['mean', 'count', 'RMS'] variables: [totalSnowDepth] groups to process: ['ombg', 'oman'] qc groups: ['EffectiveQC0', 'EffectiveQC1'] - output file: "{{ stat_current_cycle_YMDH }}.ioda_hofx_stats.{{ obspace }}.nc" + output file: "{{ snow_obsdataout_path }}/{{ snow_obsdataout_prefix }}{{ observation_from_jcb }}{{ snow_obsdataout_suffix }}" + output ascii file: "{{ snow_obsdataout_path }}/{{ observation_from_jcb }}_ioda_stats.txt" regular grid binning: bin size in degrees: 1.0 use negative longitudes: true diff --git a/parm/jcb-gdas/algorithm/obstats/snow/snocvr_template.yaml.j2 b/parm/jcb-gdas/observation_statistics/snow/snocvr.yaml.j2 similarity index 84% rename from parm/jcb-gdas/algorithm/obstats/snow/snocvr_template.yaml.j2 rename to parm/jcb-gdas/observation_statistics/snow/snocvr.yaml.j2 index 51035aa83..d90590f82 100644 --- a/parm/jcb-gdas/algorithm/obstats/snow/snocvr_template.yaml.j2 +++ b/parm/jcb-gdas/observation_statistics/snow/snocvr.yaml.j2 @@ -3,14 +3,15 @@ obsdatain: engine: type: H5File - obsfile: {{ snow_obsdatain_path }}/snow/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc + obsfile: "{{ snow_obsdatain_path }}/{{ snow_obsdatain_prefix }}{{ observation_from_jcb }}{{ snow_obsdatain_suffix }}" simulated variables: ['totalSnowDepth'] observed variables: ['totalSnowDepth'] statistics to compute: ['mean', 'count', 'RMS'] variables: [totalSnowDepth] groups to process: ['ombg', 'oman'] qc groups: ['EffectiveQC0', 'EffectiveQC1'] - output file: "{{ stat_current_cycle_YMDH }}.ioda_hofx_stats.{{ obspace }}.nc" + output file: "{{ snow_obsdataout_path }}/{{ snow_obsdataout_prefix }}{{ observation_from_jcb }}{{ snow_obsdataout_suffix }}" + output ascii file: "{{ snow_obsdataout_path }}/{{ observation_from_jcb }}_ioda_stats.txt" regular grid binning: bin size in degrees: 1.0 use negative longitudes: true diff --git a/parm/jcb-gdas/algorithm/obstats/snow/snocvr_snomad_template.yaml.j2 b/parm/jcb-gdas/observation_statistics/snow/snocvr_snomad.yaml.j2 similarity index 84% rename from parm/jcb-gdas/algorithm/obstats/snow/snocvr_snomad_template.yaml.j2 rename to parm/jcb-gdas/observation_statistics/snow/snocvr_snomad.yaml.j2 index 7eb522576..f175d6e21 100644 --- a/parm/jcb-gdas/algorithm/obstats/snow/snocvr_snomad_template.yaml.j2 +++ b/parm/jcb-gdas/observation_statistics/snow/snocvr_snomad.yaml.j2 @@ -3,14 +3,15 @@ obsdatain: engine: type: H5File - obsfile: {{ snow_obsdatain_path }}/snow/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc + obsfile: "{{ snow_obsdatain_path }}/{{ snow_obsdatain_prefix }}{{ observation_from_jcb }}{{ snow_obsdatain_suffix }}" simulated variables: ['totalSnowDepth'] observed variables: ['totalSnowDepth'] statistics to compute: ['mean', 'count', 'RMS'] variables: [totalSnowDepth] groups to process: ['ombg', 'oman'] qc groups: ['EffectiveQC0', 'EffectiveQC1'] - output file: "{{ stat_current_cycle_YMDH }}.ioda_hofx_stats.{{ obspace }}.nc" + output file: "{{ snow_obsdataout_path }}/{{ snow_obsdataout_prefix }}{{ observation_from_jcb }}{{ snow_obsdataout_suffix }}" + output ascii file: "{{ snow_obsdataout_path }}/{{ observation_from_jcb }}_ioda_stats.txt" regular grid binning: bin size in degrees: 1.0 use negative longitudes: true diff --git a/parm/jcb-gdas/algorithm/obstats/snow/snomad_template.yaml.j2 b/parm/jcb-gdas/observation_statistics/snow/snomad.yaml.j2 similarity index 84% rename from parm/jcb-gdas/algorithm/obstats/snow/snomad_template.yaml.j2 rename to parm/jcb-gdas/observation_statistics/snow/snomad.yaml.j2 index 3ecd99856..1bdde113d 100644 --- a/parm/jcb-gdas/algorithm/obstats/snow/snomad_template.yaml.j2 +++ b/parm/jcb-gdas/observation_statistics/snow/snomad.yaml.j2 @@ -3,14 +3,15 @@ obsdatain: engine: type: H5File - obsfile: {{ snow_obsdatain_path }}/snow/diag_{{ obspace }}_{{ stat_current_cycle_YMDH }}.nc + obsfile: "{{ snow_obsdatain_path }}/{{ snow_obsdatain_prefix }}{{ observation_from_jcb }}{{ snow_obsdatain_suffix }}" simulated variables: ['totalSnowDepth'] observed variables: ['totalSnowDepth'] statistics to compute: ['mean', 'count', 'RMS'] variables: [totalSnowDepth] groups to process: ['ombg', 'oman'] qc groups: ['EffectiveQC0', 'EffectiveQC1'] - output file: "{{ stat_current_cycle_YMDH }}.ioda_hofx_stats.{{ obspace }}.nc" + output file: "{{ snow_obsdataout_path }}/{{ snow_obsdataout_prefix }}{{ observation_from_jcb }}{{ snow_obsdataout_suffix }}" + output ascii file: "{{ snow_obsdataout_path }}/{{ observation_from_jcb }}_ioda_stats.txt" regular grid binning: bin size in degrees: 1.0 use negative longitudes: true From 758c46bec3253f09d52db716b0db9f8f96ee205a Mon Sep 17 00:00:00 2001 From: Dan Holdaway <27729500+danholdaway@users.noreply.github.com> Date: Fri, 2 Jan 2026 12:46:47 -0500 Subject: [PATCH 184/199] Add option to override all filters with a global setting (#213) The companion PR to https://github.com/NOAA-EMC/jcb/pull/31 offering an example of how to apply the change to just the GOES-16 ABI YAML. Co-authored-by: danholdaway Co-authored-by: Cory Martin --- .../client_integration/gdas-atmosphere-templates.yaml | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml b/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml index 3ceef304a..4e57bd759 100644 --- a/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml +++ b/parm/jcb-gdas/test/client_integration/gdas-atmosphere-templates.yaml @@ -96,6 +96,17 @@ atmosphere_final_increment_prefix: "./anl/atminc." # ------------------ observations: all_observations +# JCB level option to replace obs filters +# --------------------------------------- +replace_obs_filters: + observations: + - abi_g16 + override_filters: + - filter: PreQC + maxvalue: 1.0 + action: + name: reject + crtm_coefficient_path: "DATA/crtm/" # Naming conventions for observational files From e728e038ae113090bf48f96ffd218f925ea93a94 Mon Sep 17 00:00:00 2001 From: Jiarui Dong Date: Fri, 2 Jan 2026 12:48:47 -0500 Subject: [PATCH 185/199] Modify the GHCN obs path for snow DA (#220) Modify the GHCN obs path to use the data from the `prep` subdirectory under the running directory. This PR contributes to https://github.com/NOAA-EMC/GDASApp/pull/2019 Co-authored-by: Cory Martin --- parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 index 668c160bb..06a0184ef 100644 --- a/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 @@ -7,7 +7,7 @@ obsdatain: engine: type: H5File - obsfile: "{{snow_obsdatain_path}}/{{snow_obsdatain_prefix}}{{observation_from_jcb}}.nc" + obsfile: "{{snow_prepobs_path}}/{{snow_obsdatain_prefix}}{{observation_from_jcb}}.nc" missing file action: warn obsdataout: engine: From 909f50ee1dc15e6680775ffed8aaced621901081 Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Fri, 2 Jan 2026 13:05:04 -0500 Subject: [PATCH 186/199] Add some more conventional plus one radiance IODA-stats templates for GSI ncdiags (#222) As the title says, we are adding in conventional sfc, sfcship, sondes, and aircraft as well as ATMS N20 templates. The workflow will produce IODA files from GSI nc diags, and then these can be used in the IODA-stats utility to produce summary stats. --- ...si_sfc_ps.yaml.j2 => aircraft_gsi.yaml.j2} | 8 +-- .../atmosphere_gsi/atms_n20_gsi.yaml.j2 | 51 ++++++++++++++++++ .../atmosphere_gsi/sfc_gsi.yaml.j2 | 54 +++++++++++++++++++ .../atmosphere_gsi/sfcship_gsi.yaml.j2 | 54 +++++++++++++++++++ .../atmosphere_gsi/sondes_gsi.yaml.j2 | 54 +++++++++++++++++++ 5 files changed, 217 insertions(+), 4 deletions(-) rename parm/jcb-gdas/observation_statistics/atmosphere_gsi/{gsi_sfc_ps.yaml.j2 => aircraft_gsi.yaml.j2} (86%) create mode 100644 parm/jcb-gdas/observation_statistics/atmosphere_gsi/atms_n20_gsi.yaml.j2 create mode 100644 parm/jcb-gdas/observation_statistics/atmosphere_gsi/sfc_gsi.yaml.j2 create mode 100644 parm/jcb-gdas/observation_statistics/atmosphere_gsi/sfcship_gsi.yaml.j2 create mode 100644 parm/jcb-gdas/observation_statistics/atmosphere_gsi/sondes_gsi.yaml.j2 diff --git a/parm/jcb-gdas/observation_statistics/atmosphere_gsi/gsi_sfc_ps.yaml.j2 b/parm/jcb-gdas/observation_statistics/atmosphere_gsi/aircraft_gsi.yaml.j2 similarity index 86% rename from parm/jcb-gdas/observation_statistics/atmosphere_gsi/gsi_sfc_ps.yaml.j2 rename to parm/jcb-gdas/observation_statistics/atmosphere_gsi/aircraft_gsi.yaml.j2 index 88316c9e9..9a1f3c802 100644 --- a/parm/jcb-gdas/observation_statistics/atmosphere_gsi/gsi_sfc_ps.yaml.j2 +++ b/parm/jcb-gdas/observation_statistics/atmosphere_gsi/aircraft_gsi.yaml.j2 @@ -1,13 +1,13 @@ - obs space: - name: gsi_sfc_ps + name: aircraft_gsi obsdatain: engine: type: H5File obsfile: "{{ atmosphere_gsi_obsdatain_path }}/{{ atmosphere_gsi_obsdatain_prefix }}{{ observation_from_jcb }}{{ atmosphere_gsi_obsdatain_suffix }}" - simulated variables: ['stationPressure'] - observed variables: ['stationPressure'] + simulated variables: ['airTemperature', 'specificHumidity', 'windEastward', 'windNorthward'] + observed variables: ['airTemperature', 'specificHumidity', 'windEastward', 'windNorthward'] statistics to compute: ['mean', 'count', 'RMS'] - variables: ['stationPressure'] + variables: ['airTemperature', 'specificHumidity', 'windEastward', 'windNorthward'] groups to process: ['ombg', 'oman'] qc groups: ['GsiEffectiveQCGes', 'GsiEffectiveQCAnl'] output file: "{{ atmosphere_gsi_obsdataout_path }}/{{ atmosphere_gsi_obsdataout_prefix }}{{ observation_from_jcb }}{{ atmosphere_gsi_obsdataout_suffix }}" diff --git a/parm/jcb-gdas/observation_statistics/atmosphere_gsi/atms_n20_gsi.yaml.j2 b/parm/jcb-gdas/observation_statistics/atmosphere_gsi/atms_n20_gsi.yaml.j2 new file mode 100644 index 000000000..067b7959a --- /dev/null +++ b/parm/jcb-gdas/observation_statistics/atmosphere_gsi/atms_n20_gsi.yaml.j2 @@ -0,0 +1,51 @@ +- obs space: + name: atms_n20_gsi + obsdatain: + engine: + type: H5File + obsfile: "{{ atmosphere_gsi_obsdatain_path }}/{{ atmosphere_gsi_obsdatain_prefix }}{{ observation_from_jcb }}{{ atmosphere_gsi_obsdatain_suffix }}" + simulated variables: ['brightnessTemperature'] + observed variables: ['brightnessTemperature'] + statistics to compute: ['count', 'mean', 'RMS'] + variables: ['brightnessTemperature'] + channels: [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22] + groups to process: ['ombg', 'oman'] + qc groups: ['GsiEffectiveQCGes', 'GsiEffectiveQCAnl'] + output file: "{{ atmosphere_gsi_obsdataout_path }}/{{ atmosphere_gsi_obsdataout_prefix }}{{ observation_from_jcb }}{{ atmosphere_gsi_obsdataout_suffix }}" + output ascii file: "{{ atmosphere_gsi_obsdataout_path }}/{{ observation_from_jcb }}_ioda_stats.txt" + regular grid binning: + bin size in degrees: 2.5 + use negative longitudes: true + domains to process: + - domain: + name: "SH" + first mask variable: latitude + first mask range: [-90,0] + - domain: + name: "NH" + first mask variable: latitude + first mask range: [0,90] + - domain: + name: "CONUS" + first mask variable: latitude + first mask range: [25,49] + second mask variable: longitude + second mask range: [-125,-66] + - domain: + name: "Europe" + first mask variable: latitude + first mask range: [35,70] + second mask variable: longitude + second mask range: [-11,38] + - domain: + name: "Africa" + first mask variable: latitude + first mask range: [-35,37] + second mask variable: longitude + second mask range: [-17,52] + - domain: + name: "Asia" + first mask variable: latitude + first mask range: [0,70] + second mask variable: longitude + second mask range: [38, 180] diff --git a/parm/jcb-gdas/observation_statistics/atmosphere_gsi/sfc_gsi.yaml.j2 b/parm/jcb-gdas/observation_statistics/atmosphere_gsi/sfc_gsi.yaml.j2 new file mode 100644 index 000000000..2f6f5b7c4 --- /dev/null +++ b/parm/jcb-gdas/observation_statistics/atmosphere_gsi/sfc_gsi.yaml.j2 @@ -0,0 +1,54 @@ +- obs space: + name: sfc_gsi + obsdatain: + engine: + type: H5File + obsfile: "{{ atmosphere_gsi_obsdatain_path }}/{{ atmosphere_gsi_obsdatain_prefix }}{{ observation_from_jcb }}{{ atmosphere_gsi_obsdatain_suffix }}" + simulated variables: ['stationPressure', 'airTemperature', 'specificHumidity', 'windEastward', 'windNorthward'] + observed variables: ['stationPressure', 'airTemperature', 'specificHumidity', 'windEastward', 'windNorthward'] + statistics to compute: ['mean', 'count', 'RMS'] + variables: ['stationPressure', 'airTemperature', 'specificHumidity', 'windEastward', 'windNorthward'] + groups to process: ['ombg', 'oman'] + qc groups: ['GsiEffectiveQCGes', 'GsiEffectiveQCAnl'] + output file: "{{ atmosphere_gsi_obsdataout_path }}/{{ atmosphere_gsi_obsdataout_prefix }}{{ observation_from_jcb }}{{ atmosphere_gsi_obsdataout_suffix }}" + output ascii file: "{{ atmosphere_gsi_obsdataout_path }}/{{ observation_from_jcb }}_ioda_stats.txt" + ascii vertical bins: + vertical bin names: [1000+, 1000-900, 900-800, 800-600, 600-400, 400-300, 300-250, 250-200, 200-150, 150-100, 100-50] + vertical bin ranges: [100000, 90000, 80000, 60000, 40000, 30000, 25000, 20000, 15000, 10000, 5000] + vertical bin variable: pressure + regular grid binning: + bin size in degrees: 2.5 + use negative longitudes: true + domains to process: + - domain: + name: "SH" + first mask variable: latitude + first mask range: [-90,0] + - domain: + name: "NH" + first mask variable: latitude + first mask range: [0,90] + - domain: + name: "CONUS" + first mask variable: latitude + first mask range: [25,49] + second mask variable: longitude + second mask range: [-125,-66] + - domain: + name: "Europe" + first mask variable: latitude + first mask range: [35,70] + second mask variable: longitude + second mask range: [-11,38] + - domain: + name: "Africa" + first mask variable: latitude + first mask range: [-35,37] + second mask variable: longitude + second mask range: [-17,52] + - domain: + name: "Asia" + first mask variable: latitude + first mask range: [0,70] + second mask variable: longitude + second mask range: [38, 180] diff --git a/parm/jcb-gdas/observation_statistics/atmosphere_gsi/sfcship_gsi.yaml.j2 b/parm/jcb-gdas/observation_statistics/atmosphere_gsi/sfcship_gsi.yaml.j2 new file mode 100644 index 000000000..5341825be --- /dev/null +++ b/parm/jcb-gdas/observation_statistics/atmosphere_gsi/sfcship_gsi.yaml.j2 @@ -0,0 +1,54 @@ +- obs space: + name: sfcship_gsi + obsdatain: + engine: + type: H5File + obsfile: "{{ atmosphere_gsi_obsdatain_path }}/{{ atmosphere_gsi_obsdatain_prefix }}{{ observation_from_jcb }}{{ atmosphere_gsi_obsdatain_suffix }}" + simulated variables: ['stationPressure', 'airTemperature', 'virtualTemperature', 'specificHumidity', 'windEastward', 'windNorthward'] + observed variables: ['stationPressure', 'airTemperature', 'virtualTemperature', 'specificHumidity', 'windEastward', 'windNorthward'] + statistics to compute: ['mean', 'count', 'RMS'] + variables: ['stationPressure', 'airTemperature', 'virtualTemperature', 'specificHumidity', 'windEastward', 'windNorthward'] + groups to process: ['ombg', 'oman'] + qc groups: ['GsiEffectiveQCGes', 'GsiEffectiveQCAnl'] + output file: "{{ atmosphere_gsi_obsdataout_path }}/{{ atmosphere_gsi_obsdataout_prefix }}{{ observation_from_jcb }}{{ atmosphere_gsi_obsdataout_suffix }}" + output ascii file: "{{ atmosphere_gsi_obsdataout_path }}/{{ observation_from_jcb }}_ioda_stats.txt" + ascii vertical bins: + vertical bin names: [1000+, 1000-900, 900-800, 800-600, 600-400, 400-300, 300-250, 250-200, 200-150, 150-100, 100-50] + vertical bin ranges: [100000, 90000, 80000, 60000, 40000, 30000, 25000, 20000, 15000, 10000, 5000] + vertical bin variable: pressure + regular grid binning: + bin size in degrees: 2.5 + use negative longitudes: true + domains to process: + - domain: + name: "SH" + first mask variable: latitude + first mask range: [-90,0] + - domain: + name: "NH" + first mask variable: latitude + first mask range: [0,90] + - domain: + name: "CONUS" + first mask variable: latitude + first mask range: [25,49] + second mask variable: longitude + second mask range: [-125,-66] + - domain: + name: "Europe" + first mask variable: latitude + first mask range: [35,70] + second mask variable: longitude + second mask range: [-11,38] + - domain: + name: "Africa" + first mask variable: latitude + first mask range: [-35,37] + second mask variable: longitude + second mask range: [-17,52] + - domain: + name: "Asia" + first mask variable: latitude + first mask range: [0,70] + second mask variable: longitude + second mask range: [38, 180] diff --git a/parm/jcb-gdas/observation_statistics/atmosphere_gsi/sondes_gsi.yaml.j2 b/parm/jcb-gdas/observation_statistics/atmosphere_gsi/sondes_gsi.yaml.j2 new file mode 100644 index 000000000..533dab478 --- /dev/null +++ b/parm/jcb-gdas/observation_statistics/atmosphere_gsi/sondes_gsi.yaml.j2 @@ -0,0 +1,54 @@ +- obs space: + name: sondes_gsi + obsdatain: + engine: + type: H5File + obsfile: "{{ atmosphere_gsi_obsdatain_path }}/{{ atmosphere_gsi_obsdatain_prefix }}{{ observation_from_jcb }}{{ atmosphere_gsi_obsdatain_suffix }}" + simulated variables: ['stationPressure', 'airTemperature', 'virtualTemperature', 'specificHumidity', 'windEastward', 'windNorthward'] + observed variables: ['stationPressure', 'airTemperature', 'virtualTemperature', 'specificHumidity', 'windEastward', 'windNorthward'] + statistics to compute: ['mean', 'count', 'RMS'] + variables: ['stationPressure', 'airTemperature', 'virtualTemperature', 'specificHumidity', 'windEastward', 'windNorthward'] + groups to process: ['ombg', 'oman'] + qc groups: ['GsiEffectiveQCGes', 'GsiEffectiveQCAnl'] + output file: "{{ atmosphere_gsi_obsdataout_path }}/{{ atmosphere_gsi_obsdataout_prefix }}{{ observation_from_jcb }}{{ atmosphere_gsi_obsdataout_suffix }}" + output ascii file: "{{ atmosphere_gsi_obsdataout_path }}/{{ observation_from_jcb }}_ioda_stats.txt" + ascii vertical bins: + vertical bin names: [1000+, 1000-900, 900-800, 800-600, 600-400, 400-300, 300-250, 250-200, 200-150, 150-100, 100-50] + vertical bin ranges: [100000, 90000, 80000, 60000, 40000, 30000, 25000, 20000, 15000, 10000, 5000] + vertical bin variable: pressure + regular grid binning: + bin size in degrees: 2.5 + use negative longitudes: true + domains to process: + - domain: + name: "SH" + first mask variable: latitude + first mask range: [-90,0] + - domain: + name: "NH" + first mask variable: latitude + first mask range: [0,90] + - domain: + name: "CONUS" + first mask variable: latitude + first mask range: [25,49] + second mask variable: longitude + second mask range: [-125,-66] + - domain: + name: "Europe" + first mask variable: latitude + first mask range: [35,70] + second mask variable: longitude + second mask range: [-11,38] + - domain: + name: "Africa" + first mask variable: latitude + first mask range: [-35,37] + second mask variable: longitude + second mask range: [-17,52] + - domain: + name: "Asia" + first mask variable: latitude + first mask range: [0,70] + second mask variable: longitude + second mask range: [38, 180] From 05e9bd27d0c9b3bf08f07f56c67a7bdfbeca1928 Mon Sep 17 00:00:00 2001 From: Jiarui Dong Date: Fri, 2 Jan 2026 13:08:06 -0500 Subject: [PATCH 187/199] Change the suffix to the input IMS data (#221) Update the input file suffixes from `.asc` to `.dat`, including those for ASCII and NetCDF formats. Contributes to https://github.com/NOAA-EMC/GDASApp/pull/2023 --------- Co-authored-by: Cory Martin --- parm/jcb-gdas/algorithm/snow/snow_ims_scf_preprocess.yaml.j2 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/parm/jcb-gdas/algorithm/snow/snow_ims_scf_preprocess.yaml.j2 b/parm/jcb-gdas/algorithm/snow/snow_ims_scf_preprocess.yaml.j2 index 67ac7276a..7d5f4e9e0 100644 --- a/parm/jcb-gdas/algorithm/snow/snow_ims_scf_preprocess.yaml.j2 +++ b/parm/jcb-gdas/algorithm/snow/snow_ims_scf_preprocess.yaml.j2 @@ -24,7 +24,7 @@ geometry: filename_orog: '{{ snow_orog_prefix }}_oro_data.nc' date: '{{ snow_background_time_iso }}' output ioda file: '{{snow_obsdatain_path}}/{{snow_obsdatain_prefix}}ims_snow.tm00.nc' -input scf file: './obs/ims{{ snow_background_time_julian }}_4km_v1.3.asc' +input scf file: './obs/ims{{ snow_background_time_julian }}_4km_v1.3.{{ ims_scf_obs_suffix }}' mapping file: './obs/IMS4km_to_FV3_mapping.{{ snow_orog_prefix }}_oro_data.nc' variables: - totalSnowDepth From 28d1f672179fcfdbc26c06b1208da8effb3dc9a8 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Thu, 8 Jan 2026 09:51:45 -0500 Subject: [PATCH 188/199] Replace hardcoded observation names with {{observation_from_jcb}} template variable (#223) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The `name` field in observation templates was hardcoded, requiring updates in both the observation key and the template when names changed. Now uses the `{{observation_from_jcb}}` variable that JCB algorithm templates already reference. ## Changes - Updated `obs space: name:` field in 101 observation templates across all domains (aero, atmosphere, atmosphere-lgetkf, marine, snow, soil) - Replaced hardcoded strings like `"IASI METOP-A"` with `{{observation_from_jcb}}` **Before:** ```yaml obs space: name: IASI METOP-A obsdatain: engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" ``` **After:** ```yaml obs space: name: {{observation_from_jcb}} obsdatain: engine: type: H5File obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" ``` The rendered name now matches the observation key used throughout the configuration, eliminating dual maintenance.
Original prompt > > ---- > > *This section details on the original issue you should resolve* > > The "name" key in an observation template should not be hardcoded > The `name` key in the observations templates should not be hardcoded, but should rather be `{{observation_from_jcb}}` which corresponds to the observation index variable used in JCB algorithm templates. This make for one more parameter to change when the name of an observation changes. > > anywhere you see name: in a yaml.j2 file should be {{ observation_from_jcb }} instead of whatever it currently is defined as > > ## Comments on the Issue (you are @copilot in this section) > > > >
- Fixes NOAA-EMC/jcb-gdas#217 --- ✨ Let Copilot coding agent [set things up for you](https://github.com/NOAA-EMC/jcb-gdas/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot) — coding agent works faster and does higher quality work when set up for your repo. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: CoryMartin-NOAA <6354668+CoryMartin-NOAA@users.noreply.github.com> --- parm/jcb-gdas/observations/aero/retrieval_aod_viirs_n20.yaml.j2 | 2 +- parm/jcb-gdas/observations/aero/retrieval_aod_viirs_n21.yaml.j2 | 2 +- parm/jcb-gdas/observations/aero/retrieval_aod_viirs_npp.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 | 2 +- .../observations/atmosphere-lgetkf/ascatw.ascat_metop-b.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere-lgetkf/atms_n20.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere-lgetkf/atms_npp.yaml.j2 | 2 +- .../observations/atmosphere-lgetkf/conventional_ps.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere-lgetkf/gnssro.yaml.j2 | 2 +- .../observations/atmosphere-lgetkf/gnssro_cosmic2.yaml.j2 | 2 +- .../observations/atmosphere-lgetkf/iasi_metop-a.yaml.j2 | 2 +- .../observations/atmosphere-lgetkf/iasi_metop-b.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere-lgetkf/ompsnp_npp.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere-lgetkf/ompstc_npp.yaml.j2 | 2 +- .../observations/atmosphere-lgetkf/ozone.ompsnp_npp.yaml.j2 | 2 +- .../observations/atmosphere-lgetkf/ozone.ompstc_npp.yaml.j2 | 2 +- .../observations/atmosphere-lgetkf/satwnd.abi_goes-16.yaml.j2 | 2 +- .../observations/atmosphere-lgetkf/satwnd.abi_goes-18.yaml.j2 | 2 +- .../observations/atmosphere-lgetkf/satwnd.ahi_h8.yaml.j2 | 2 +- .../observations/atmosphere-lgetkf/satwnd.leogeo_multi.yaml.j2 | 2 +- .../observations/atmosphere-lgetkf/satwnd.viirs_n20.yaml.j2 | 2 +- .../observations/atmosphere-lgetkf/satwnd.viirs_npp.yaml.j2 | 2 +- .../atmosphere-lgetkf/scatwnd.ascat_metop-b.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere-lgetkf/sondes.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/abi_g16.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/abi_g17.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/aircraft_humidity.yaml.j2 | 2 +- .../observations/atmosphere/aircraft_temperature.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/aircraft_wind.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/amsr2_gcom-w1.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/cris-fsr_n21.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/cris-fsr_npp.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/gnssro.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/gnssro_cosmic2.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/gnssro_geoopt.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/gnssro_grace.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/gnssro_k5.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/gnssro_metop.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/gnssro_paz.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/gnssro_piq.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/gnssro_s6.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/gnssro_spire.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/gnssro_tdm.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/gnssro_tsx.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/iasi_metop-c.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/ozone.omi_aura.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_n20.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_npp.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/ozone.ompstc_n20.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/ozone.ompstc_npp.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/prepbufr_adpsfc.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/prepbufr_adpupa.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/prepbufr_sfcshp.yaml.j2 | 2 +- .../jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 | 2 +- .../jcb-gdas/observations/atmosphere/satwnd.abi_goes-17.yaml.j2 | 2 +- .../jcb-gdas/observations/atmosphere/satwnd.abi_goes-18.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h8.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h9.yaml.j2 | 2 +- .../observations/atmosphere/satwnd.avhrr_metop-a.yaml.j2 | 2 +- .../observations/atmosphere/satwnd.avhrr_metop-b.yaml.j2 | 2 +- .../observations/atmosphere/satwnd.avhrr_metop-c.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n15.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n18.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n19.yaml.j2 | 2 +- .../observations/atmosphere/satwnd.leogeo_multi.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/satwnd.modis_aqua.yaml.j2 | 2 +- .../jcb-gdas/observations/atmosphere/satwnd.modis_terra.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m10.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m11.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m8.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m9.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/satwnd.viirs_n20.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/satwnd.viirs_npp.yaml.j2 | 2 +- .../observations/atmosphere/scatwnd.ascat_metop-a.yaml.j2 | 2 +- .../observations/atmosphere/scatwnd.ascat_metop-b.yaml.j2 | 2 +- .../observations/atmosphere/scatwnd.ascat_metop-c.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/seviri_m08.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/seviri_m11.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/sfc.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/sfcship.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 | 2 +- parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 | 2 +- parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 | 2 +- parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 | 2 +- parm/jcb-gdas/observations/snow/ims_snow.yaml.j2 | 2 +- parm/jcb-gdas/observations/snow/madis_snow.yaml.j2 | 2 +- parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 | 2 +- parm/jcb-gdas/observations/snow/snocvr.yaml.j2 | 2 +- parm/jcb-gdas/observations/snow/snocvr_snomad.yaml.j2 | 2 +- parm/jcb-gdas/observations/snow/snomad.yaml.j2 | 2 +- parm/jcb-gdas/observations/soil/smap_soil.yaml.j2 | 2 +- 101 files changed, 101 insertions(+), 101 deletions(-) diff --git a/parm/jcb-gdas/observations/aero/retrieval_aod_viirs_n20.yaml.j2 b/parm/jcb-gdas/observations/aero/retrieval_aod_viirs_n20.yaml.j2 index 9b3c9a7b6..378358946 100644 --- a/parm/jcb-gdas/observations/aero/retrieval_aod_viirs_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/aero/retrieval_aod_viirs_n20.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: retrieval_aod_viirs_n20 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/aero/retrieval_aod_viirs_n21.yaml.j2 b/parm/jcb-gdas/observations/aero/retrieval_aod_viirs_n21.yaml.j2 index be8475a21..efa9cdd5c 100644 --- a/parm/jcb-gdas/observations/aero/retrieval_aod_viirs_n21.yaml.j2 +++ b/parm/jcb-gdas/observations/aero/retrieval_aod_viirs_n21.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: retrieval_aod_viirs_n21 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/aero/retrieval_aod_viirs_npp.yaml.j2 b/parm/jcb-gdas/observations/aero/retrieval_aod_viirs_npp.yaml.j2 index 831eea4ae..119aaf18d 100644 --- a/parm/jcb-gdas/observations/aero/retrieval_aod_viirs_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/aero/retrieval_aod_viirs_npp.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: retrieval_aod_viirs_npp + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml.j2 index a99271332..2690123e3 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/aircraft.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: Aircraft + name: {{observation_from_jcb}} distribution: name: "{{distribution_type}}" halo size: 1250e3 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 index 17ed934c7..de4116299 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: AMSUA N19 + name: {{observation_from_jcb}} distribution: name: "{{distribution_type}}" halo size: 1250e3 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/ascatw.ascat_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/ascatw.ascat_metop-b.yaml.j2 index 831fb80d7..8a9afdee7 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/ascatw.ascat_metop-b.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/ascatw.ascat_metop-b.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: ascatw_ascat_metop-b + name: {{observation_from_jcb}} distribution: name: "{{distribution_type}}" halo size: 1250e3 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/atms_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/atms_n20.yaml.j2 index 6c5754f4c..a2c0b5f4e 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/atms_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/atms_n20.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: ATMS N20 + name: {{observation_from_jcb}} distribution: name: "{{distribution_type}}" halo size: 1250e3 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/atms_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/atms_npp.yaml.j2 index ea17f1241..2842cae62 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/atms_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/atms_npp.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: ATMS NPP + name: {{observation_from_jcb}} distribution: name: "{{distribution_type}}" halo size: 1250e3 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml.j2 index 54b3241d3..a9be15805 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/conventional_ps.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: surface_ps + name: {{observation_from_jcb}} distribution: name: "{{distribution_type}}" halo size: 1250e3 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/gnssro.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/gnssro.yaml.j2 index 9da6bd594..df980564e 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/gnssro.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/gnssro.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: gnssrobndnbam + name: {{observation_from_jcb}} distribution: name: "{{distribution_type}}" halo size: 1250e3 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/gnssro_cosmic2.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/gnssro_cosmic2.yaml.j2 index 17d3437bf..8651b324e 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/gnssro_cosmic2.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/gnssro_cosmic2.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: gnssrobndnbam_cosmic2 + name: {{observation_from_jcb}} distribution: name: "{{distribution_type}}" halo size: 1250e3 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/iasi_metop-a.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/iasi_metop-a.yaml.j2 index 9edba023a..e82979e41 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/iasi_metop-a.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/iasi_metop-a.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: IASI METOP-A + name: {{observation_from_jcb}} distribution: name: "{{distribution_type}}" halo size: 1250e3 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/iasi_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/iasi_metop-b.yaml.j2 index d22bdbcfa..50796f517 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/iasi_metop-b.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/iasi_metop-b.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: IASI METOP-B + name: {{observation_from_jcb}} distribution: name: "{{distribution_type}}" halo size: 1250e3 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/ompsnp_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/ompsnp_npp.yaml.j2 index 45c49bae1..1c347b498 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/ompsnp_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/ompsnp_npp.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: ompsnp_npp + name: {{observation_from_jcb}} distribution: name: "{{distribution_type}}" halo size: 1250e3 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/ompstc_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/ompstc_npp.yaml.j2 index 523946cda..2ece37515 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/ompstc_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/ompstc_npp.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: ompstc_npp + name: {{observation_from_jcb}} distribution: name: "{{distribution_type}}" halo size: 1250e3 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/ozone.ompsnp_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/ozone.ompsnp_npp.yaml.j2 index bb003b8a0..ff078f37c 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/ozone.ompsnp_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/ozone.ompsnp_npp.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: ompsnp_npp + name: {{observation_from_jcb}} distribution: name: "{{distribution_type}}" halo size: 1250e3 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/ozone.ompstc_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/ozone.ompstc_npp.yaml.j2 index 78904b9fc..a6ed761e3 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/ozone.ompstc_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/ozone.ompstc_npp.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: ompstc_npp + name: {{observation_from_jcb}} distribution: name: "{{distribution_type}}" halo size: 1250e3 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-16.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-16.yaml.j2 index d3b214ca8..8434d44ee 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-16.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-16.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: satwind_goes-16 + name: {{observation_from_jcb}} distribution: name: "{{distribution_type}}" halo size: 1250e3 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-18.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-18.yaml.j2 index ebbc86b02..b8dabb474 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-18.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-18.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: satwind_goes-18 + name: {{observation_from_jcb}} distribution: name: "{{distribution_type}}" halo size: 1250e3 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.ahi_h8.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.ahi_h8.yaml.j2 index e9ee771c3..fc609d96b 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.ahi_h8.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.ahi_h8.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: satwind_ahi_h8 + name: {{observation_from_jcb}} distribution: name: "{{distribution_type}}" halo size: 1250e3 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.leogeo_multi.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.leogeo_multi.yaml.j2 index 0558d66bb..c5e2b0497 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.leogeo_multi.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.leogeo_multi.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: satwind_leogeo_multi + name: {{observation_from_jcb}} distribution: name: "{{distribution_type}}" halo size: 1250e3 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.viirs_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.viirs_n20.yaml.j2 index 2c5f1b66e..f7d577a15 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.viirs_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.viirs_n20.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: satwind_viirs_n20 + name: {{observation_from_jcb}} distribution: name: "{{distribution_type}}" halo size: 1250e3 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.viirs_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.viirs_npp.yaml.j2 index 18998d58d..7dd263716 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.viirs_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.viirs_npp.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: satwind_viirs_npp + name: {{observation_from_jcb}} distribution: name: "{{distribution_type}}" halo size: 1250e3 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/scatwnd.ascat_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/scatwnd.ascat_metop-b.yaml.j2 index 5a63e4310..224521fb9 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/scatwnd.ascat_metop-b.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/scatwnd.ascat_metop-b.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: ascatw_ascat_metop-b + name: {{observation_from_jcb}} distribution: name: "{{distribution_type}}" halo size: 1250e3 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/sondes.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/sondes.yaml.j2 index dea76f050..3064438f6 100644 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/sondes.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere-lgetkf/sondes.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: sondes + name: {{observation_from_jcb}} distribution: name: "{{distribution_type}}" halo size: 1250e3 diff --git a/parm/jcb-gdas/observations/atmosphere/abi_g16.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/abi_g16.yaml.j2 index 441d53889..0c721f322 100644 --- a/parm/jcb-gdas/observations/atmosphere/abi_g16.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/abi_g16.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: abi_g16 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/abi_g17.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/abi_g17.yaml.j2 index beae2197d..fc972cb54 100644 --- a/parm/jcb-gdas/observations/atmosphere/abi_g17.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/abi_g17.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: abi_g17 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/aircraft_humidity.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/aircraft_humidity.yaml.j2 index 1f30ddb31..07807fe0e 100644 --- a/parm/jcb-gdas/observations/atmosphere/aircraft_humidity.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/aircraft_humidity.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: Aircraft + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/aircraft_temperature.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/aircraft_temperature.yaml.j2 index 09129d45b..8c6e03d6b 100644 --- a/parm/jcb-gdas/observations/atmosphere/aircraft_temperature.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/aircraft_temperature.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: Aircraft + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/aircraft_wind.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/aircraft_wind.yaml.j2 index 0063393ff..03c4b78e8 100644 --- a/parm/jcb-gdas/observations/atmosphere/aircraft_wind.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/aircraft_wind.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: Aircraft + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/amsr2_gcom-w1.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/amsr2_gcom-w1.yaml.j2 index 32759dbe4..6618ea029 100644 --- a/parm/jcb-gdas/observations/atmosphere/amsr2_gcom-w1.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/amsr2_gcom-w1.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: AMSR2 GCOM-W1 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 index fe0698b5d..541ee0880 100644 --- a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: AMSUA N19 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 index 6b8492d15..bebdb2757 100644 --- a/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: ATMS N20 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 index 1a39ca6e6..b5c0860d4 100644 --- a/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: ATMS N21 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 index e55ee691f..ee80fbce1 100644 --- a/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: ATMS NPP + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 index 5f7ffaff8..f870262ad 100644 --- a/parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/conventional_ps.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: surface_ps + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 index b59b83db9..e590b8bf5 100644 --- a/parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: CRIS-FSR N20 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/cris-fsr_n21.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/cris-fsr_n21.yaml.j2 index aa91769a7..d6e5fa189 100644 --- a/parm/jcb-gdas/observations/atmosphere/cris-fsr_n21.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/cris-fsr_n21.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: CRIS-FSR N21 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/cris-fsr_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/cris-fsr_npp.yaml.j2 index 8a0e8fb23..486d231a0 100644 --- a/parm/jcb-gdas/observations/atmosphere/cris-fsr_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/cris-fsr_npp.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: CRIS-FSR NPP + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro.yaml.j2 index a57f604ae..ed5355799 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: gnssrobndnbam + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_cosmic2.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_cosmic2.yaml.j2 index e18033dd4..f4dd7c9ad 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_cosmic2.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_cosmic2.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: gnssrobndnbam_cosmic2 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_geoopt.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_geoopt.yaml.j2 index 1ac81847d..bc4988048 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_geoopt.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_geoopt.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: gnssrobndnbam_geoopt + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_grace.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_grace.yaml.j2 index 8f4f4ad7a..5e77ca283 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_grace.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_grace.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: gnssrobndnbam_grace + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_k5.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_k5.yaml.j2 index 6a7e60c23..5e77ca283 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_k5.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_k5.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: gnssrobndnbam_k5 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_metop.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_metop.yaml.j2 index be786d247..a3754e23d 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_metop.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_metop.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: gnssrobndnbam_metop + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_paz.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_paz.yaml.j2 index 861115fc2..366df08e2 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_paz.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_paz.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: gnssrobndnbam_paz + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_piq.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_piq.yaml.j2 index 03f89dd43..bc4988048 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_piq.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_piq.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: gnssrobndnbam_piq + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_s6.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_s6.yaml.j2 index 614f4a1ac..5e77ca283 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_s6.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_s6.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: gnssrobndnbam_s6 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_spire.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_spire.yaml.j2 index 6cff8d0fe..bc4988048 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_spire.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_spire.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: gnssrobndnbam_spire + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_tdm.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_tdm.yaml.j2 index 448fd2889..5e77ca283 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_tdm.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_tdm.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: gnssrobndnbam_tdm + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_tsx.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_tsx.yaml.j2 index 2539e6888..5e77ca283 100644 --- a/parm/jcb-gdas/observations/atmosphere/gnssro_tsx.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/gnssro_tsx.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: gnssrobndnbam_tsx + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 index 155d5acd2..08e14f433 100644 --- a/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: IASI METOP-A + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 index 1133d2bb2..49f48c546 100644 --- a/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: IASI METOP-B + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/iasi_metop-c.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/iasi_metop-c.yaml.j2 index 965e2b55d..2dab46e68 100644 --- a/parm/jcb-gdas/observations/atmosphere/iasi_metop-c.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/iasi_metop-c.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: IASI METOP-C + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/ozone.omi_aura.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ozone.omi_aura.yaml.j2 index 99a31c703..d9bed021d 100644 --- a/parm/jcb-gdas/observations/atmosphere/ozone.omi_aura.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ozone.omi_aura.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: omi_aura + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_n20.yaml.j2 index 98e855e5c..ddba90cbb 100644 --- a/parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_n20.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: ompsnp_n20 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_npp.yaml.j2 index b1e70e6cd..ddba90cbb 100644 --- a/parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_npp.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: ompsnp_npp + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/ozone.ompstc_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ozone.ompstc_n20.yaml.j2 index 6639fa7df..ac791421c 100644 --- a/parm/jcb-gdas/observations/atmosphere/ozone.ompstc_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ozone.ompstc_n20.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: ompstc_n20 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/ozone.ompstc_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ozone.ompstc_npp.yaml.j2 index e20cf11ed..ac791421c 100644 --- a/parm/jcb-gdas/observations/atmosphere/ozone.ompstc_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ozone.ompstc_npp.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: ompstc_npp + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/prepbufr_adpsfc.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/prepbufr_adpsfc.yaml.j2 index c68693a24..d11297ebf 100644 --- a/parm/jcb-gdas/observations/atmosphere/prepbufr_adpsfc.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/prepbufr_adpsfc.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: surface_ps + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/prepbufr_adpupa.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/prepbufr_adpupa.yaml.j2 index 91c7f9b9a..7d6c7190d 100644 --- a/parm/jcb-gdas/observations/atmosphere/prepbufr_adpupa.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/prepbufr_adpupa.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: surface_ps + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/prepbufr_sfcshp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/prepbufr_sfcshp.yaml.j2 index c68693a24..d11297ebf 100644 --- a/parm/jcb-gdas/observations/atmosphere/prepbufr_sfcshp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/prepbufr_sfcshp.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: surface_ps + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 index 1ba275681..e6ddcd6c9 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: satwind_goes-16 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-17.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-17.yaml.j2 index a24e094b0..e6ddcd6c9 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-17.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-17.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: satwind_goes-17 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-18.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-18.yaml.j2 index e775b87ee..e6ddcd6c9 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-18.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-18.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: satwind_goes-18 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h8.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h8.yaml.j2 index c39792cbc..9ec989945 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h8.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h8.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: satwind_ahi_h8 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h9.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h9.yaml.j2 index 467014e79..9ec989945 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h9.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h9.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: satwind_ahi_h9 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-a.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-a.yaml.j2 index 1528a38d3..d95a8b0c9 100755 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-a.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-a.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: satwind_avhrr_metop-a + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-b.yaml.j2 index 5c18ce8ff..d95a8b0c9 100755 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-b.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-b.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: satwind_avhrr_metop-b + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-c.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-c.yaml.j2 index 2edcf7b3f..d95a8b0c9 100755 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-c.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-c.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: satwind_avhrr_metop-c + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n15.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n15.yaml.j2 index 373200745..d95a8b0c9 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n15.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n15.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: satwind_avhrr_n15 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n18.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n18.yaml.j2 index c29900be6..d95a8b0c9 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n18.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n18.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: satwind_avhrr_n18 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n19.yaml.j2 index 3b549d5c6..d95a8b0c9 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n19.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n19.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: satwind_avhrr_n19 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.leogeo_multi.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.leogeo_multi.yaml.j2 index 3a396b7b7..1aaa0fda0 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.leogeo_multi.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.leogeo_multi.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: satwind_leogeo_multi + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.modis_aqua.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.modis_aqua.yaml.j2 index f4ebcb18b..0971bf8fe 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.modis_aqua.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.modis_aqua.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: satwind_modis_aqua + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.modis_terra.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.modis_terra.yaml.j2 index f8722f4b2..434a0f671 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.modis_terra.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.modis_terra.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: satwind_modis_terra + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m10.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m10.yaml.j2 index 94b5207a7..11306468d 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m10.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m10.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: satwind_seviri_m10 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m11.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m11.yaml.j2 index e105a14c1..afb74befc 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m11.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m11.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: satwind_seviri_m11 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m8.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m8.yaml.j2 index d32c4dad4..0135ab4fd 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m8.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m8.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: satwind_seviri_m8 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m9.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m9.yaml.j2 index d0ff40087..b7e6b094a 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m9.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m9.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: satwind_seviri_m9 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_n20.yaml.j2 index 5984fe02b..5fc941577 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_n20.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: satwind_viirs_n20 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_npp.yaml.j2 index 96cc59ebf..5fc941577 100644 --- a/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_npp.yaml.j2 @@ -1,5 +1,5 @@ - obs space: - name: satwind_viirs_npp + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-a.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-a.yaml.j2 index 3af852f3f..f7eb43944 100644 --- a/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-a.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-a.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: ascatw_ascat_metop-a + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-b.yaml.j2 index 286631f6a..f7eb43944 100644 --- a/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-b.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-b.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: ascatw_ascat_metop-b + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-c.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-c.yaml.j2 index 86f00cd9f..f7eb43944 100644 --- a/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-c.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-c.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: ascatw_ascat_metop-c + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/seviri_m08.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/seviri_m08.yaml.j2 index ac826f86f..11a3221c6 100644 --- a/parm/jcb-gdas/observations/atmosphere/seviri_m08.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/seviri_m08.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: seviri_m08 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/seviri_m11.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/seviri_m11.yaml.j2 index 4fe5bad46..c8b3c8bfc 100644 --- a/parm/jcb-gdas/observations/atmosphere/seviri_m11.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/seviri_m11.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: seviri_m11 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/sfc.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/sfc.yaml.j2 index 0848cc06e..d6c4775ad 100644 --- a/parm/jcb-gdas/observations/atmosphere/sfc.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/sfc.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: sfc + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/sfcship.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/sfcship.yaml.j2 index 9da9a6124..60d0b19bc 100644 --- a/parm/jcb-gdas/observations/atmosphere/sfcship.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/sfcship.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: sfcship + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 index 7ffccbb73..e708798fc 100644 --- a/parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: sondes + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 index f81b13d07..7cb077bf1 100644 --- a/parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: SSMIS F17 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 index b5578d55b..9b99a5c64 100644 --- a/parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: SSMIS F18 + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 index 834e3b37a..cd3aa5f8a 100644 --- a/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/adpsfc_snow.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: adpsfc_snow + name: {{observation_from_jcb}} obsdatain: engine: type: bufr diff --git a/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 index 06a0184ef..4ad6a7444 100644 --- a/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/ghcn_snow.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: ghcn_snow + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/snow/ims_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/ims_snow.yaml.j2 index 1f1b8e77c..ef1685c3e 100644 --- a/parm/jcb-gdas/observations/snow/ims_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/ims_snow.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: ims_snow + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/snow/madis_snow.yaml.j2 b/parm/jcb-gdas/observations/snow/madis_snow.yaml.j2 index 23339c78c..b900fd37b 100644 --- a/parm/jcb-gdas/observations/snow/madis_snow.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/madis_snow.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: madis_snow + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 b/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 index d1683eca1..a465cbe3e 100644 --- a/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/sfcsno.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: sfcsno + name: {{observation_from_jcb}} obsdatain: engine: type: bufr diff --git a/parm/jcb-gdas/observations/snow/snocvr.yaml.j2 b/parm/jcb-gdas/observations/snow/snocvr.yaml.j2 index 7785cf1a1..0626a145b 100644 --- a/parm/jcb-gdas/observations/snow/snocvr.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/snocvr.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: snocvr + name: {{observation_from_jcb}} obsdatain: engine: type: bufr diff --git a/parm/jcb-gdas/observations/snow/snocvr_snomad.yaml.j2 b/parm/jcb-gdas/observations/snow/snocvr_snomad.yaml.j2 index 865d9ae2d..9d9d64825 100644 --- a/parm/jcb-gdas/observations/snow/snocvr_snomad.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/snocvr_snomad.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: snocvr_snomad + name: {{observation_from_jcb}} obsdatain: engine: type: H5File diff --git a/parm/jcb-gdas/observations/snow/snomad.yaml.j2 b/parm/jcb-gdas/observations/snow/snomad.yaml.j2 index cccb13bdd..58a23ac10 100644 --- a/parm/jcb-gdas/observations/snow/snomad.yaml.j2 +++ b/parm/jcb-gdas/observations/snow/snomad.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: snomad + name: {{observation_from_jcb}} obsdatain: engine: type: bufr diff --git a/parm/jcb-gdas/observations/soil/smap_soil.yaml.j2 b/parm/jcb-gdas/observations/soil/smap_soil.yaml.j2 index ff7e9a575..760e5344e 100644 --- a/parm/jcb-gdas/observations/soil/smap_soil.yaml.j2 +++ b/parm/jcb-gdas/observations/soil/smap_soil.yaml.j2 @@ -3,7 +3,7 @@ # Observation Space (I/O) # ----------------------- obs space: - name: smap_soil + name: {{observation_from_jcb}} obsdatain: engine: type: H5File From c9a357151ae6b21561f26a4d13ce8aee5d8c64a2 Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Fri, 9 Jan 2026 09:06:24 -0500 Subject: [PATCH 189/199] Start to make obs space names consistent (#207) First step towards consolidating around a naming convention and making sure that YAML filename == obs space name --------- Co-authored-by: RussTreadon-NOAA --- .../{abi_g16.yaml => radiance_abi_g16.yaml} | 0 .../{abi_g17.yaml => radiance_abi_g17.yaml} | 0 ...om-w1.yaml => radiance_amsr2_gcom-w1.yaml} | 0 ...amsua_n19.yaml => radiance_amsua_n19.yaml} | 0 .../{atms_n20.yaml => radiance_atms_n20.yaml} | 0 .../{atms_n21.yaml => radiance_atms_n21.yaml} | 0 .../{atms_npp.yaml => radiance_atms_npp.yaml} | 0 ...sr_n20.yaml => radiance_cris-fsr_n20.yaml} | 0 ...sr_n21.yaml => radiance_cris-fsr_n21.yaml} | 0 ...sr_npp.yaml => radiance_cris-fsr_npp.yaml} | 0 ...etop-a.yaml => radiance_iasi_metop-a.yaml} | 0 ...etop-b.yaml => radiance_iasi_metop-b.yaml} | 0 ...etop-c.yaml => radiance_iasi_metop-c.yaml} | 0 ...viri_m08.yaml => radiance_seviri_m08.yaml} | 0 ...viri_m11.yaml => radiance_seviri_m11.yaml} | 0 ...ssmis_f17.yaml => radiance_ssmis_f17.yaml} | 0 ...ssmis_f18.yaml => radiance_ssmis_f18.yaml} | 0 .../atmosphere-lgetkf/ompsnp_npp.yaml.j2 | 339 ------------------ .../atmosphere-lgetkf/ompstc_npp.yaml.j2 | 144 -------- ...n19.yaml.j2 => radiance_amsua_n19.yaml.j2} | 0 ..._n20.yaml.j2 => radiance_atms_n20.yaml.j2} | 0 ..._npp.yaml.j2 => radiance_atms_npp.yaml.j2} | 0 ....yaml.j2 => radiance_iasi_metop-a.yaml.j2} | 0 ....yaml.j2 => radiance_iasi_metop-b.yaml.j2} | 0 ...l.j2 => retrieval_amv_abi_goes-16.yaml.j2} | 0 ...l.j2 => retrieval_amv_abi_goes-18.yaml.j2} | 0 ...8.yaml.j2 => retrieval_amv_ahi_h8.yaml.j2} | 0 ....j2 => retrieval_amv_leogeo_multi.yaml.j2} | 0 ...aml.j2 => retrieval_amv_viirs_n20.yaml.j2} | 0 ...aml.j2 => retrieval_amv_viirs_npp.yaml.j2} | 0 ...j2 => retrieval_osw_ascat_metop-b.yaml.j2} | 0 ....j2 => retrieval_ozone_ompsnp_npp.yaml.j2} | 0 ....j2 => retrieval_ozone_ompstc_npp.yaml.j2} | 0 .../scatwnd.ascat_metop-b.yaml.j2 | 325 ----------------- ...i_g16.yaml.j2 => radiance_abi_g16.yaml.j2} | 0 ...i_g17.yaml.j2 => radiance_abi_g17.yaml.j2} | 0 ...yaml.j2 => radiance_amsr2_gcom-w1.yaml.j2} | 0 ...n19.yaml.j2 => radiance_amsua_n19.yaml.j2} | 0 ..._n20.yaml.j2 => radiance_atms_n20.yaml.j2} | 0 ..._n21.yaml.j2 => radiance_atms_n21.yaml.j2} | 0 ..._npp.yaml.j2 => radiance_atms_npp.yaml.j2} | 0 ....yaml.j2 => radiance_cris-fsr_n20.yaml.j2} | 0 ....yaml.j2 => radiance_cris-fsr_n21.yaml.j2} | 0 ....yaml.j2 => radiance_cris-fsr_npp.yaml.j2} | 0 ....yaml.j2 => radiance_iasi_metop-a.yaml.j2} | 0 ....yaml.j2 => radiance_iasi_metop-b.yaml.j2} | 0 ....yaml.j2 => radiance_iasi_metop-c.yaml.j2} | 0 ...08.yaml.j2 => radiance_seviri_m08.yaml.j2} | 0 ...11.yaml.j2 => radiance_seviri_m11.yaml.j2} | 0 ...f17.yaml.j2 => radiance_ssmis_f17.yaml.j2} | 0 ...f18.yaml.j2 => radiance_ssmis_f18.yaml.j2} | 0 ...l.j2 => retrieval_amv_abi_goes-16.yaml.j2} | 0 ...l.j2 => retrieval_amv_abi_goes-17.yaml.j2} | 0 ...l.j2 => retrieval_amv_abi_goes-18.yaml.j2} | 0 ...8.yaml.j2 => retrieval_amv_ahi_h8.yaml.j2} | 0 ...9.yaml.j2 => retrieval_amv_ahi_h9.yaml.j2} | 0 ...j2 => retrieval_amv_avhrr_metop-a.yaml.j2} | 0 ...j2 => retrieval_amv_avhrr_metop-b.yaml.j2} | 0 ...j2 => retrieval_amv_avhrr_metop-c.yaml.j2} | 0 ...aml.j2 => retrieval_amv_avhrr_n15.yaml.j2} | 0 ...aml.j2 => retrieval_amv_avhrr_n18.yaml.j2} | 0 ...aml.j2 => retrieval_amv_avhrr_n19.yaml.j2} | 0 ....j2 => retrieval_amv_leogeo_multi.yaml.j2} | 0 ...ml.j2 => retrieval_amv_modis_aqua.yaml.j2} | 0 ...l.j2 => retrieval_amv_modis_terra.yaml.j2} | 0 ...ml.j2 => retrieval_amv_seviri_m10.yaml.j2} | 0 ...ml.j2 => retrieval_amv_seviri_m11.yaml.j2} | 0 ...aml.j2 => retrieval_amv_seviri_m8.yaml.j2} | 0 ...aml.j2 => retrieval_amv_seviri_m9.yaml.j2} | 0 ...aml.j2 => retrieval_amv_viirs_n20.yaml.j2} | 0 ...aml.j2 => retrieval_amv_viirs_npp.yaml.j2} | 0 ...j2 => retrieval_osw_ascat_metop-a.yaml.j2} | 0 ...j2 => retrieval_osw_ascat_metop-b.yaml.j2} | 0 ...j2 => retrieval_osw_ascat_metop-c.yaml.j2} | 0 ...ml.j2 => retrieval_ozone_omi_aura.yaml.j2} | 0 ....j2 => retrieval_ozone_ompsnp_n20.yaml.j2} | 0 ....j2 => retrieval_ozone_ompsnp_npp.yaml.j2} | 0 ....j2 => retrieval_ozone_ompstc_n20.yaml.j2} | 0 ....j2 => retrieval_ozone_ompstc_npp.yaml.j2} | 0 79 files changed, 808 deletions(-) rename parm/jcb-gdas/observation_chronicle/atmosphere/{abi_g16.yaml => radiance_abi_g16.yaml} (100%) rename parm/jcb-gdas/observation_chronicle/atmosphere/{abi_g17.yaml => radiance_abi_g17.yaml} (100%) rename parm/jcb-gdas/observation_chronicle/atmosphere/{amsr2_gcom-w1.yaml => radiance_amsr2_gcom-w1.yaml} (100%) rename parm/jcb-gdas/observation_chronicle/atmosphere/{amsua_n19.yaml => radiance_amsua_n19.yaml} (100%) rename parm/jcb-gdas/observation_chronicle/atmosphere/{atms_n20.yaml => radiance_atms_n20.yaml} (100%) rename parm/jcb-gdas/observation_chronicle/atmosphere/{atms_n21.yaml => radiance_atms_n21.yaml} (100%) rename parm/jcb-gdas/observation_chronicle/atmosphere/{atms_npp.yaml => radiance_atms_npp.yaml} (100%) rename parm/jcb-gdas/observation_chronicle/atmosphere/{cris-fsr_n20.yaml => radiance_cris-fsr_n20.yaml} (100%) rename parm/jcb-gdas/observation_chronicle/atmosphere/{cris-fsr_n21.yaml => radiance_cris-fsr_n21.yaml} (100%) rename parm/jcb-gdas/observation_chronicle/atmosphere/{cris-fsr_npp.yaml => radiance_cris-fsr_npp.yaml} (100%) rename parm/jcb-gdas/observation_chronicle/atmosphere/{iasi_metop-a.yaml => radiance_iasi_metop-a.yaml} (100%) rename parm/jcb-gdas/observation_chronicle/atmosphere/{iasi_metop-b.yaml => radiance_iasi_metop-b.yaml} (100%) rename parm/jcb-gdas/observation_chronicle/atmosphere/{iasi_metop-c.yaml => radiance_iasi_metop-c.yaml} (100%) rename parm/jcb-gdas/observation_chronicle/atmosphere/{seviri_m08.yaml => radiance_seviri_m08.yaml} (100%) rename parm/jcb-gdas/observation_chronicle/atmosphere/{seviri_m11.yaml => radiance_seviri_m11.yaml} (100%) rename parm/jcb-gdas/observation_chronicle/atmosphere/{ssmis_f17.yaml => radiance_ssmis_f17.yaml} (100%) rename parm/jcb-gdas/observation_chronicle/atmosphere/{ssmis_f18.yaml => radiance_ssmis_f18.yaml} (100%) delete mode 100644 parm/jcb-gdas/observations/atmosphere-lgetkf/ompsnp_npp.yaml.j2 delete mode 100644 parm/jcb-gdas/observations/atmosphere-lgetkf/ompstc_npp.yaml.j2 rename parm/jcb-gdas/observations/atmosphere-lgetkf/{amsua_n19.yaml.j2 => radiance_amsua_n19.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere-lgetkf/{atms_n20.yaml.j2 => radiance_atms_n20.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere-lgetkf/{atms_npp.yaml.j2 => radiance_atms_npp.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere-lgetkf/{iasi_metop-a.yaml.j2 => radiance_iasi_metop-a.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere-lgetkf/{iasi_metop-b.yaml.j2 => radiance_iasi_metop-b.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere-lgetkf/{satwnd.abi_goes-16.yaml.j2 => retrieval_amv_abi_goes-16.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere-lgetkf/{satwnd.abi_goes-18.yaml.j2 => retrieval_amv_abi_goes-18.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere-lgetkf/{satwnd.ahi_h8.yaml.j2 => retrieval_amv_ahi_h8.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere-lgetkf/{satwnd.leogeo_multi.yaml.j2 => retrieval_amv_leogeo_multi.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere-lgetkf/{satwnd.viirs_n20.yaml.j2 => retrieval_amv_viirs_n20.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere-lgetkf/{satwnd.viirs_npp.yaml.j2 => retrieval_amv_viirs_npp.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere-lgetkf/{ascatw.ascat_metop-b.yaml.j2 => retrieval_osw_ascat_metop-b.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere-lgetkf/{ozone.ompsnp_npp.yaml.j2 => retrieval_ozone_ompsnp_npp.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere-lgetkf/{ozone.ompstc_npp.yaml.j2 => retrieval_ozone_ompstc_npp.yaml.j2} (100%) delete mode 100644 parm/jcb-gdas/observations/atmosphere-lgetkf/scatwnd.ascat_metop-b.yaml.j2 rename parm/jcb-gdas/observations/atmosphere/{abi_g16.yaml.j2 => radiance_abi_g16.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{abi_g17.yaml.j2 => radiance_abi_g17.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{amsr2_gcom-w1.yaml.j2 => radiance_amsr2_gcom-w1.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{amsua_n19.yaml.j2 => radiance_amsua_n19.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{atms_n20.yaml.j2 => radiance_atms_n20.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{atms_n21.yaml.j2 => radiance_atms_n21.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{atms_npp.yaml.j2 => radiance_atms_npp.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{cris-fsr_n20.yaml.j2 => radiance_cris-fsr_n20.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{cris-fsr_n21.yaml.j2 => radiance_cris-fsr_n21.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{cris-fsr_npp.yaml.j2 => radiance_cris-fsr_npp.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{iasi_metop-a.yaml.j2 => radiance_iasi_metop-a.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{iasi_metop-b.yaml.j2 => radiance_iasi_metop-b.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{iasi_metop-c.yaml.j2 => radiance_iasi_metop-c.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{seviri_m08.yaml.j2 => radiance_seviri_m08.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{seviri_m11.yaml.j2 => radiance_seviri_m11.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{ssmis_f17.yaml.j2 => radiance_ssmis_f17.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{ssmis_f18.yaml.j2 => radiance_ssmis_f18.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{satwnd.abi_goes-16.yaml.j2 => retrieval_amv_abi_goes-16.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{satwnd.abi_goes-17.yaml.j2 => retrieval_amv_abi_goes-17.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{satwnd.abi_goes-18.yaml.j2 => retrieval_amv_abi_goes-18.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{satwnd.ahi_h8.yaml.j2 => retrieval_amv_ahi_h8.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{satwnd.ahi_h9.yaml.j2 => retrieval_amv_ahi_h9.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{satwnd.avhrr_metop-a.yaml.j2 => retrieval_amv_avhrr_metop-a.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{satwnd.avhrr_metop-b.yaml.j2 => retrieval_amv_avhrr_metop-b.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{satwnd.avhrr_metop-c.yaml.j2 => retrieval_amv_avhrr_metop-c.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{satwnd.avhrr_n15.yaml.j2 => retrieval_amv_avhrr_n15.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{satwnd.avhrr_n18.yaml.j2 => retrieval_amv_avhrr_n18.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{satwnd.avhrr_n19.yaml.j2 => retrieval_amv_avhrr_n19.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{satwnd.leogeo_multi.yaml.j2 => retrieval_amv_leogeo_multi.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{satwnd.modis_aqua.yaml.j2 => retrieval_amv_modis_aqua.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{satwnd.modis_terra.yaml.j2 => retrieval_amv_modis_terra.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{satwnd.seviri_m10.yaml.j2 => retrieval_amv_seviri_m10.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{satwnd.seviri_m11.yaml.j2 => retrieval_amv_seviri_m11.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{satwnd.seviri_m8.yaml.j2 => retrieval_amv_seviri_m8.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{satwnd.seviri_m9.yaml.j2 => retrieval_amv_seviri_m9.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{satwnd.viirs_n20.yaml.j2 => retrieval_amv_viirs_n20.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{satwnd.viirs_npp.yaml.j2 => retrieval_amv_viirs_npp.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{scatwnd.ascat_metop-a.yaml.j2 => retrieval_osw_ascat_metop-a.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{scatwnd.ascat_metop-b.yaml.j2 => retrieval_osw_ascat_metop-b.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{scatwnd.ascat_metop-c.yaml.j2 => retrieval_osw_ascat_metop-c.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{ozone.omi_aura.yaml.j2 => retrieval_ozone_omi_aura.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{ozone.ompsnp_n20.yaml.j2 => retrieval_ozone_ompsnp_n20.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{ozone.ompsnp_npp.yaml.j2 => retrieval_ozone_ompsnp_npp.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{ozone.ompstc_n20.yaml.j2 => retrieval_ozone_ompstc_n20.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{ozone.ompstc_npp.yaml.j2 => retrieval_ozone_ompstc_npp.yaml.j2} (100%) diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/abi_g16.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/radiance_abi_g16.yaml similarity index 100% rename from parm/jcb-gdas/observation_chronicle/atmosphere/abi_g16.yaml rename to parm/jcb-gdas/observation_chronicle/atmosphere/radiance_abi_g16.yaml diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/abi_g17.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/radiance_abi_g17.yaml similarity index 100% rename from parm/jcb-gdas/observation_chronicle/atmosphere/abi_g17.yaml rename to parm/jcb-gdas/observation_chronicle/atmosphere/radiance_abi_g17.yaml diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/amsr2_gcom-w1.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/radiance_amsr2_gcom-w1.yaml similarity index 100% rename from parm/jcb-gdas/observation_chronicle/atmosphere/amsr2_gcom-w1.yaml rename to parm/jcb-gdas/observation_chronicle/atmosphere/radiance_amsr2_gcom-w1.yaml diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/amsua_n19.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/radiance_amsua_n19.yaml similarity index 100% rename from parm/jcb-gdas/observation_chronicle/atmosphere/amsua_n19.yaml rename to parm/jcb-gdas/observation_chronicle/atmosphere/radiance_amsua_n19.yaml diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/atms_n20.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/radiance_atms_n20.yaml similarity index 100% rename from parm/jcb-gdas/observation_chronicle/atmosphere/atms_n20.yaml rename to parm/jcb-gdas/observation_chronicle/atmosphere/radiance_atms_n20.yaml diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/atms_n21.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/radiance_atms_n21.yaml similarity index 100% rename from parm/jcb-gdas/observation_chronicle/atmosphere/atms_n21.yaml rename to parm/jcb-gdas/observation_chronicle/atmosphere/radiance_atms_n21.yaml diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/atms_npp.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/radiance_atms_npp.yaml similarity index 100% rename from parm/jcb-gdas/observation_chronicle/atmosphere/atms_npp.yaml rename to parm/jcb-gdas/observation_chronicle/atmosphere/radiance_atms_npp.yaml diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/cris-fsr_n20.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/radiance_cris-fsr_n20.yaml similarity index 100% rename from parm/jcb-gdas/observation_chronicle/atmosphere/cris-fsr_n20.yaml rename to parm/jcb-gdas/observation_chronicle/atmosphere/radiance_cris-fsr_n20.yaml diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/cris-fsr_n21.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/radiance_cris-fsr_n21.yaml similarity index 100% rename from parm/jcb-gdas/observation_chronicle/atmosphere/cris-fsr_n21.yaml rename to parm/jcb-gdas/observation_chronicle/atmosphere/radiance_cris-fsr_n21.yaml diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/cris-fsr_npp.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/radiance_cris-fsr_npp.yaml similarity index 100% rename from parm/jcb-gdas/observation_chronicle/atmosphere/cris-fsr_npp.yaml rename to parm/jcb-gdas/observation_chronicle/atmosphere/radiance_cris-fsr_npp.yaml diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/iasi_metop-a.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/radiance_iasi_metop-a.yaml similarity index 100% rename from parm/jcb-gdas/observation_chronicle/atmosphere/iasi_metop-a.yaml rename to parm/jcb-gdas/observation_chronicle/atmosphere/radiance_iasi_metop-a.yaml diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/iasi_metop-b.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/radiance_iasi_metop-b.yaml similarity index 100% rename from parm/jcb-gdas/observation_chronicle/atmosphere/iasi_metop-b.yaml rename to parm/jcb-gdas/observation_chronicle/atmosphere/radiance_iasi_metop-b.yaml diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/iasi_metop-c.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/radiance_iasi_metop-c.yaml similarity index 100% rename from parm/jcb-gdas/observation_chronicle/atmosphere/iasi_metop-c.yaml rename to parm/jcb-gdas/observation_chronicle/atmosphere/radiance_iasi_metop-c.yaml diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/seviri_m08.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/radiance_seviri_m08.yaml similarity index 100% rename from parm/jcb-gdas/observation_chronicle/atmosphere/seviri_m08.yaml rename to parm/jcb-gdas/observation_chronicle/atmosphere/radiance_seviri_m08.yaml diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/seviri_m11.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/radiance_seviri_m11.yaml similarity index 100% rename from parm/jcb-gdas/observation_chronicle/atmosphere/seviri_m11.yaml rename to parm/jcb-gdas/observation_chronicle/atmosphere/radiance_seviri_m11.yaml diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/ssmis_f17.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/radiance_ssmis_f17.yaml similarity index 100% rename from parm/jcb-gdas/observation_chronicle/atmosphere/ssmis_f17.yaml rename to parm/jcb-gdas/observation_chronicle/atmosphere/radiance_ssmis_f17.yaml diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/ssmis_f18.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/radiance_ssmis_f18.yaml similarity index 100% rename from parm/jcb-gdas/observation_chronicle/atmosphere/ssmis_f18.yaml rename to parm/jcb-gdas/observation_chronicle/atmosphere/radiance_ssmis_f18.yaml diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/ompsnp_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/ompsnp_npp.yaml.j2 deleted file mode 100644 index 1c347b498..000000000 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/ompsnp_npp.yaml.j2 +++ /dev/null @@ -1,339 +0,0 @@ -- - - # Observation Space (I/O) - # ----------------------- - obs space: - name: {{observation_from_jcb}} - distribution: - name: "{{distribution_type}}" - halo size: 1250e3 - obsdatain: - engine: - type: H5File - obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" - obsgrouping: - group variables: ["latitude"] - sort variable: "pressure" - sort order: "ascending" - obsdataout: - engine: - type: H5File - obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" - io pool: - max pool size: 1 - simulated variables: [ozoneLayer] - - #obs operator: - # name: AtmVertInterpLay - # geovals: [mole_fraction_of_ozone_in_air] - # coefficients: [0.007886131] # convert from ppmv to DU - # nlevels: [22] - - # Observation Operator - # -------------------- - obs operator: - name: ColumnRetrieval - nlayers_retrieval: 1 - tracer variables: [mole_fraction_of_ozone_in_air] - isApriori: false - isAveragingKernel: false - totalNoVertice: false - stretchVertices: none #options: top, bottom, topbottom, none - # model units coeff: 2.240013904035E-3 # this number to match the gsihofx values - model units coeff: 2.241398632746E-3 # this number to match the gsihofx values (use this) - # the actual scientific conversion factor is - # 2.1415E-3 kg[O3]/m-2 to DU - # so the name of the geovals - # is also likely wrong, as it apprears to be a mass - # fraction given the conversion factor needed - - # Observation Pre Filters (QC) - # ---------------------------- - obs pre filters: - # Observation error assignment - - filter: Perform Action - filter variables: - - name: ozoneLayer - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [0.001, 10.1325, 16.00935, 25.43258, 40.32735, 63.93607, 101.325, - 160.0935, 254.3257, 403.2735, 639.3608, 1013.25, 1600.935, 2543.258, 4032.735, - 6393.607, 10132.5, 16009.35, 25432.57, 40327.35, 63936.07, 101325] - errors: [7.7236, 0.020, 0.020, 0.025, 0.080, 0.150, 0.056, 0.125, 0.200, - 0.299, 0.587, 0.864, 1.547, 2.718, 3.893, 4.353, 3.971, 4.407, 4.428, - 3.312, 2.198, 2.285] - # errors: [7.7236, 0.020, 0.020, 0.025, 0.040, 0.080, 0.156, 0.245, 0.510, 1.098, 3.917, 6.124, 6.347, 5.798, 6.843, 9.253,10.091,10.967, 8.478, 5.572, 2.638, 3.525] # operational from gfs.v16.3.9 (late 2023) - - # Observation Prior Filters (QC) - # ------------------------------ - obs prior filters: - # Do not assimilation where pressure is zero - # Zero pressure indicates the data is total column ozone - - filter: RejectList - filter variables: - - name: ozoneLayer - where: - - variable: - name: MetaData/pressure - maxvalue: 0.0001 - - # Sanity check on observaton values - - filter: Bounds Check - filter variables: - - name: ozoneLayer - minvalue: 0 - maxvalue: 1000 - action: - name: reject - - # Total Ozone Quality Check (keeps 0, 2) - # 0 indentifies good data - # 2 identifies good data with a solar zenith angle > 84 degrees - - filter: RejectList - filter variables: - - name: ozoneLayer - where: - - variable: - name: MetaData/totalOzoneQuality - is_not_in: 0, 2 - - # Profile Ozone Quality Check (keeps 0, 1, 7) - # 0 : good data - # 1 : good data with a solar zenith angle > 84 degrees - # 7 : profile for which stray light correction applied - - filter: RejectList - filter variables: - - name: ozoneLayer - where: - - variable: - name: MetaData/profileOzoneQuality - is_not_in: 0, 1, 7 - - # Observation Post Filters (QC) - # ----------------------------- - obs post filters: - # Gross error check - - filter: Background Check - filter variables: - - name: ozoneLayer - absolute threshold: 120 - action: - name: reject - where: - - variable: - name: MetaData/pressure - maxvalue: 0.001 - - - filter: Background Check - filter variables: - - name: ozoneLayer - absolute threshold: 30 - action: - name: reject - where: - - variable: - name: MetaData/pressure - minvalue: 30000.0 - maxvalue: 110000.0 - - - filter: Background Check - filter variables: - - name: ozoneLayer - absolute threshold: 40 - action: - name: reject - where: - - variable: - name: MetaData/pressure - minvalue: 20000.0 - maxvalue: 30000.0 - - - filter: Background Check - filter variables: - - name: ozoneLayer - absolute threshold: 44.42 - action: - name: reject - where: - - variable: - name: MetaData/pressure - minvalue: 10100.0 - maxvalue: 20000.0 - - - filter: Background Check - filter variables: - - name: ozoneLayer - absolute threshold: 57.52 - action: - name: reject - where: - - variable: - name: MetaData/pressure - minvalue: 6400.0 - maxvalue: 10100.0 - - - filter: Background Check - filter variables: - - name: ozoneLayer - absolute threshold: 69.4 - action: - name: reject - where: - - variable: - name: MetaData/pressure - minvalue: 4000.0 - maxvalue: 6400.0 - - - filter: Background Check - filter variables: - - name: ozoneLayer - absolute threshold: 70 - action: - name: reject - where: - - variable: - name: MetaData/pressure - minvalue: 2600.0 - maxvalue: 4000.0 - - - filter: Background Check - filter variables: - - name: ozoneLayer - absolute threshold: 62.73 - action: - name: reject - where: - - variable: - name: MetaData/pressure - minvalue: 1600.0 - maxvalue: 2600.0 - - - filter: Background Check - filter variables: - - name: ozoneLayer - absolute threshold: 50.52 - action: - name: reject - where: - - variable: - name: MetaData/pressure - minvalue: 1100.0 - maxvalue: 1600.0 - - - filter: Background Check - filter variables: - - name: ozoneLayer - absolute threshold: 35.9 - action: - name: reject - where: - - variable: - name: MetaData/pressure - minvalue: 700.0 - maxvalue: 1100.0 - - - filter: Background Check - filter variables: - - name: ozoneLayer - absolute threshold: 26.41 - action: - name: reject - where: - - variable: - name: MetaData/pressure - minvalue: 400.0 - maxvalue: 700.0 - - - filter: Background Check - filter variables: - - name: ozoneLayer - absolute threshold: 20.51 - action: - name: reject - where: - - variable: - name: MetaData/pressure - minvalue: 300.0 - maxvalue: 400.0 - - - filter: Background Check - filter variables: - - name: ozoneLayer - absolute threshold: 12.82 - action: - name: reject - where: - - variable: - name: MetaData/pressure - minvalue: 200.0 - maxvalue: 300.0 - - - filter: Background Check - filter variables: - - name: ozoneLayer - absolute threshold: 10 - action: - name: reject - where: - - variable: - name: MetaData/pressure - minvalue: 70.0 - maxvalue: 200.0 - - - filter: Background Check - filter variables: - - name: ozoneLayer - absolute threshold: 5 - action: - name: reject - where: - - variable: - name: MetaData/pressure - minvalue: 40.0 - maxvalue: 70.0 - - - filter: Background Check - filter variables: - - name: ozoneLayer - absolute threshold: 2 - action: - name: reject - where: - - variable: - name: MetaData/pressure - minvalue: 30.0 - maxvalue: 40.0 - - - filter: Background Check - filter variables: - - name: ozoneLayer - absolute threshold: 1 - action: - name: reject - where: - - variable: - name: MetaData/pressure - maxvalue: 30.0 - - # End of Filters - - # Observation Localizations (LocalEnsembleDA) - # ------------------------------------------- - obs localizations: - - localization method: Horizontal Gaspari-Cohn - lengthscale: 1250e3 - max nobs: 10000 - - # GeoVaLs for Driving Observation Operators (testing mode) - # -------------------------------------------------------- - geovals: - filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" - - # Passed benchmark for UFO testing - # -------------------------------- - passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/ompstc_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/ompstc_npp.yaml.j2 deleted file mode 100644 index 2ece37515..000000000 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/ompstc_npp.yaml.j2 +++ /dev/null @@ -1,144 +0,0 @@ -- - - # Observation Space (I/O) - # ----------------------- - obs space: - name: {{observation_from_jcb}} - distribution: - name: "{{distribution_type}}" - halo size: 1250e3 - obsdatain: - engine: - type: H5File - obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" - obsdataout: - engine: - type: H5File - obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" - io pool: - max pool size: 1 - simulated variables: [ozoneTotal] - - #obs operator: - # name: AtmVertInterpLay - # geovals: [mole_fraction_of_ozone_in_air] - # coefficients: [0.007886131] # convert from ppmv to DU - # nlevels: [1] - - # Observation Operator - # -------------------- - obs operator: - name: ColumnRetrieval - nlayers_retrieval: 1 - tracer variables: [mole_fraction_of_ozone_in_air] - isApriori: false - isAveragingKernel: false - totalNoVertice: true - stretchVertices: topbottom # options: top, bottom, topbottom, none - model units coeff: 2.241398632746E-3 - - # Observation Pre Filters (QC) - # ---------------------------- - obs pre filters: - - filter: Perform Action - filter variables: - - name: ozoneTotal - action: - name: assign error - error parameter: 6.0 - - # Observation Prior Filters (QC) - # ------------------------------ - obs prior filters: - # GSI read routine QC - # range sanity check - - filter: Bounds Check - filter variables: - - name: ozoneTotal - minvalue: 0 - maxvalue: 1000 - action: - name: reject - - #- filter: Gaussian Thinning - # horizontal_mesh: 150 - # use_reduced_horizontal_grid: true - # distance_norm: geodesic - # action: - # name: reject - - # Accept total_ozone_error_flag values of 0 and 1, but not any others. - - filter: RejectList - filter variables: - - name: ozoneTotal - where: - - variable: - name: MetaData/totalOzoneQualityCode - is_not_in: 0, 1 - - - filter: RejectList - filter variables: - - name: ozoneTotal - where: - - variable: - name: MetaData/bestOzoneAlgorithmFlag - is_in: 3, 13 - - # GSI setup routine QC - - filter: RejectList - filter variables: - - name: ozoneTotal - where: - - variable: - name: MetaData/sensorScanPosition - is_in: 1, 2, 3, 4, 35 - - variable: - name: MetaData/latitude - minvalue: 50.0 - - - filter: RejectList - filter variables: - - name: ozoneTotal - where: - - variable: - name: MetaData/sensorScanPosition - is_in: 1, 2, 3, 4, 35 - - variable: - name: MetaData/latitude - maxvalue: -50.0 - - - filter: Gaussian Thinning - horizontal_mesh: 150 - use_reduced_horizontal_grid: true - distance_norm: geodesic - action: - name: reject - - # Observation Post Filters (QC) - # ----------------------------- - obs post filters: - - filter: Background Check - filter variables: - - name: ozoneTotal - threshold: 10.0 - absolute threshold: 300.0 - action: - name: reject - - # End of Filters - - # Observation Localizations (LocalEnsembleDA) - # ------------------------------------------- - obs localizations: - - localization method: Horizontal Gaspari-Cohn - lengthscale: 1250e3 - max nobs: 10000 - - # GeoVaLs for Driving Observation Operators (testing mode) - # -------------------------------------------------------- - geovals: - filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" - - # Passed benchmark for UFO testing - # -------------------------------- - passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/radiance_amsua_n19.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere-lgetkf/amsua_n19.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere-lgetkf/radiance_amsua_n19.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/atms_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/radiance_atms_n20.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere-lgetkf/atms_n20.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere-lgetkf/radiance_atms_n20.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/atms_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/radiance_atms_npp.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere-lgetkf/atms_npp.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere-lgetkf/radiance_atms_npp.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/iasi_metop-a.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/radiance_iasi_metop-a.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere-lgetkf/iasi_metop-a.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere-lgetkf/radiance_iasi_metop-a.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/iasi_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/radiance_iasi_metop-b.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere-lgetkf/iasi_metop-b.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere-lgetkf/radiance_iasi_metop-b.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-16.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/retrieval_amv_abi_goes-16.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-16.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere-lgetkf/retrieval_amv_abi_goes-16.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-18.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/retrieval_amv_abi_goes-18.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.abi_goes-18.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere-lgetkf/retrieval_amv_abi_goes-18.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.ahi_h8.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/retrieval_amv_ahi_h8.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.ahi_h8.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere-lgetkf/retrieval_amv_ahi_h8.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.leogeo_multi.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/retrieval_amv_leogeo_multi.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.leogeo_multi.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere-lgetkf/retrieval_amv_leogeo_multi.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.viirs_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/retrieval_amv_viirs_n20.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.viirs_n20.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere-lgetkf/retrieval_amv_viirs_n20.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.viirs_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/retrieval_amv_viirs_npp.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere-lgetkf/satwnd.viirs_npp.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere-lgetkf/retrieval_amv_viirs_npp.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/ascatw.ascat_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/retrieval_osw_ascat_metop-b.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere-lgetkf/ascatw.ascat_metop-b.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere-lgetkf/retrieval_osw_ascat_metop-b.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/ozone.ompsnp_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/retrieval_ozone_ompsnp_npp.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere-lgetkf/ozone.ompsnp_npp.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere-lgetkf/retrieval_ozone_ompsnp_npp.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/ozone.ompstc_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/retrieval_ozone_ompstc_npp.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere-lgetkf/ozone.ompstc_npp.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere-lgetkf/retrieval_ozone_ompstc_npp.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere-lgetkf/scatwnd.ascat_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere-lgetkf/scatwnd.ascat_metop-b.yaml.j2 deleted file mode 100644 index 224521fb9..000000000 --- a/parm/jcb-gdas/observations/atmosphere-lgetkf/scatwnd.ascat_metop-b.yaml.j2 +++ /dev/null @@ -1,325 +0,0 @@ -- - - # Observation Space (I/O) - # ----------------------- - obs space: - name: {{observation_from_jcb}} - distribution: - name: "{{distribution_type}}" - halo size: 1250e3 - obsdatain: - engine: - type: H5File - obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" - obsdataout: - engine: - type: H5File - obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" - io pool: - max pool size: 1 - simulated variables: [windEastward, windNorthward] - - # Observation Operator - # -------------------- - obs operator: - name: VertInterp - # Use height vertical coordinate first - # vertical coordinate: geometric_height - vertical coordinate: geopotential_height - observation vertical coordinate group: DerivedVariables - observation vertical coordinate: adjustedHeight - interpolation method: linear - hofx scaling field: SurfaceWindScalingHeight - hofx scaling field group: DerivedVariables - - # Linear Observation Operator - # --------------------------- - linear obs operator: - name: VertInterp - vertical coordinate: geopotential_height - observation vertical coordinate group: DerivedVariables - observation vertical coordinate: adjustedHeight - interpolation method: linear - - # Observation Pre Filters (QC) - # ---------------------------- - obs pre filters: - - filter: Gaussian Thinning - horizontal_mesh: 75 - use_reduced_horizontal_grid: true - round_horizontal_bin_count_to_nearest: true - partition_longitude_bins_using_mesh: true - action: - name: reduce obs space - - # Observation Prior Filters (QC) - # ------------------------------ - obs prior filters: - # Apply variable changes needed for rescaled height coordinate - - filter: Variable Transforms - Transform: AdjustedHeightCoordinate - SkipWhenNoObs: false - - # Apply variable changes needed for wind scaling - - filter: Variable Transforms - Transform: SurfaceWindScalingHeight - SkipWhenNoObs: false - - # Assign the initial observation error (constant value, 1.5 m/s right now). - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - action: - name: assign error - error parameter: 1.5 - - # Calculate error inflation factor for duplicate observations - #- filter: Variable Assignment - # assignments: - # - name: ObsErrorFactorDuplicateCheck/windEastward - # type: float - # function: - # name: ObsFunction/ObsErrorFactorDuplicateCheck - # options: - # use_air_pressure: true - # variable: windEastward - #- filter: Variable Assignment - # assignments: - # - name: ObsErrorFactorDuplicateCheck/windNorthward - # type: float - # function: - # name: ObsFunction/ObsErrorFactorDuplicateCheck - # options: - # use_air_pressure: true - # variable: windNorthward - # Reject all obs with PreQC mark already set above 3 - # NOTE: All scatwinds have an automatic PreQC mark of 2 (hard-wired default from GSI) - # - filter: PreQC - # maxvalue: 3 - # action: - # name: reject - - # Observation Post Filters (QC) - # ----------------------------- - obs post filters: - # Reject all ASCAT (Type 290) winds with tsavg <= 273.0 (surface temperature) - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 290 - # - variable: GeoVaLs/surface_temperature - - variable: GeoVaLs/skin_temperature_at_surface_where_land - maxvalue: 273. - action: - name: reject - - # Reject all ASCAT (Type 290) winds with isflg /= 0 (non-water surface) - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 290 - - variable: GeoVaLs/water_area_fraction - maxvalue: 0.99 - action: - name: reject - - # Reject ASCAT (Type 290) when observed component deviates from background by more than 5.0 m/s - # NOTE: This check can reject a u- or v-component of the same observation independently, which - # is fundamentally different from how GSI rejects obs (both components are rejected if - # either component fails a check). - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - test variables: - - name: ObsFunction/Arithmetic - options: - variables: - - name: ObsValue/windEastward - - name: HofX/windEastward - coefs: [1.0, -1.0] - minvalue: -5.0 - maxvalue: 5.0 - - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - test variables: - - name: ObsFunction/Arithmetic - options: - variables: - - name: ObsValue/windNorthward - - name: HofX/windNorthward - coefs: [1.0, -1.0] - minvalue: -5.0 - maxvalue: 5.0 - - # Reject OSCAT (Type 291) when observed component deviates from background by more than 6.0 m/s - # NOTE: This check can reject a u- or v-component of the same observation independently, which - # is fundamentally different from how GSI rejects obs (both components are rejected if - # either component fails a check). - - filter: Background Check - filter variables: - - name: windEastward - - name: windNorthward - threshold: 6. - absolute threshold: 6. - where: - - variable: ObsType/windEastward - is_in: 291 - action: - name: reject - - # Reject ASCAT (Type 290) when ambiguity check fails (returned value is negative) - - filter: Bounds Check - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: ObsType/windEastward - is_in: 290 - test variables: - - name: ObsFunction/ScatWindsAmbiguityCheck - options: - minimum_uv: 0.0001 # hard-coding a minimum-uv for transparancy, want this to basically be zero - maxvalue: 0. - action: - name: reject - - # All scatwinds must adjust errors based on ObsErrorFactorPressureCheck. - # This check will inflate errors for obs that are too close to either - # the model top or bottom. - - filter: Perform Action - filter variables: - - name: windEastward - where: - - variable: - name: ObsType/windEastward - is_in: 290-291 - action: - name: inflate error - inflation variable: - name: ObsFunction/ObsErrorFactorPressureCheck - options: - variable: windEastward - inflation factor: 4.0 - - - filter: Perform Action - filter variables: - - name: windNorthward - where: - - variable: - name: ObsType/windNorthward - is_in: 290-291 - action: - name: inflate error - inflation variable: - name: ObsFunction/ObsErrorFactorPressureCheck - options: - variable: windNorthward - inflation factor: 4.0 - - # All scatwinds subject to a gross error check. This is contained within - # the WindsSPDBCheck, although it is not exclusive to satwinds. - - filter: Background Check - filter variables: - - name: windEastward - function absolute threshold: - - name: ObsFunction/WindsSPDBCheck - options: - wndtype: [290, 291] - cgross: [5.0, 5.0] - error_min: [1.4, 1.4] - error_max: [6.1, 6.1] - variable: windEastward - action: - name: reject - - - filter: Background Check - filter variables: - - name: windNorthward - function absolute threshold: - - name: ObsFunction/WindsSPDBCheck - options: - wndtype: [290, 291] - cgross: [5.0, 5.0] - error_min: [1.4, 1.4] - error_max: [6.1, 6.1] - variable: windNorthward - action: - name: reject - - # The last error inflation check is for duplicate observations. This one needs - # to come last, because we don"t want to inflate errors for duplication if one - # of the duplicates should be rejected. - #- filter: Perform Action - # filter variables: - # - name: windEastward - # action: - # name: inflate error - # inflation variable: - # name: ObsErrorFactorDuplicateCheck/windEastward - #- filter: Perform Action - # filter variables: - # - name: windNorthward - # action: - # name: inflate error - # inflation variable: - # name: ObsErrorFactorDuplicateCheck/windNorthward - # There is no across-the-board inflation for nvqc=.true. for scatwinds, presumably because for - # this inflation to take place both nvqc must be .true. AND ibeta must be >0, see: - # https://github.com/NOAA-EMC/GSI/blob/14ae595af1b03471287d322596d35c0665336e95/src/gsi/setupw.f90#L1229 - # GSI settings must have ibeta>0 for satwinds, but not for scatwinds. - # - # If the ibeta settings for scatwinds were to change while nvqc remained .true., we would extend YAML to - # an additional filter that inflates final ob-errors across-the-board by 1/0.8 = 1.25. NOTE: the nvqc setting - # is defaulted to .false. in GSI code, but is overridden in global operational configuration. See: - # configuration, see: https://github.com/NOAA-EMC/global-workflow/blob/d5ae3328fa4041b177357b1133f6b92e81c859d7/scripts/exglobal_atmos_analysis.sh#L750 - # This setting activates Line 1229 of setupw.f90 to scale ratio_errors by 0.8, which is applied in - # the denominator of the final ob-error, so 1/0.8 = 1.25 factor of ob-error inflation. - # - # If this functionality were to be activated for scatwinds, you would want to include this last inflation filter. - #- filter: Perform Action - # filter variables: - # - name: windEastward - # where: - # - variable: ObsType/windEastward - # is_in: 290-291 - # action: - # name: inflate error - # inflation factor: 1.25 - #- filter: Perform Action - # filter variables: - # - name: windNorthward - # where: - # - variable: ObsType/windNorthward - # is_in: 290-291 - # action: - # name: inflate error - # inflation factor: 1.25 - # END OF FILTERS# - - - # Observation Localizations (LocalEnsembleDA) - # ------------------------------------------- - obs localizations: - - localization method: Horizontal Gaspari-Cohn - lengthscale: 1250e3 - max nobs: 10000 - - # GeoVaLs for Driving Observation Operators (testing mode) - # -------------------------------------------------------- - geovals: - filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" - - # Passed benchmark for UFO testing - # -------------------------------- - passedBenchmark: 0 diff --git a/parm/jcb-gdas/observations/atmosphere/abi_g16.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/radiance_abi_g16.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/abi_g16.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/radiance_abi_g16.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/abi_g17.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/radiance_abi_g17.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/abi_g17.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/radiance_abi_g17.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/amsr2_gcom-w1.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/radiance_amsr2_gcom-w1.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/amsr2_gcom-w1.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/radiance_amsr2_gcom-w1.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/radiance_amsua_n19.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/amsua_n19.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/radiance_amsua_n19.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/radiance_atms_n20.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/atms_n20.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/radiance_atms_n20.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/radiance_atms_n21.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/atms_n21.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/radiance_atms_n21.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/radiance_atms_npp.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/atms_npp.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/radiance_atms_npp.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/radiance_cris-fsr_n20.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/cris-fsr_n20.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/radiance_cris-fsr_n20.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/cris-fsr_n21.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/radiance_cris-fsr_n21.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/cris-fsr_n21.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/radiance_cris-fsr_n21.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/cris-fsr_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/radiance_cris-fsr_npp.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/cris-fsr_npp.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/radiance_cris-fsr_npp.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/radiance_iasi_metop-a.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/iasi_metop-a.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/radiance_iasi_metop-a.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/radiance_iasi_metop-b.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/iasi_metop-b.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/radiance_iasi_metop-b.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/iasi_metop-c.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/radiance_iasi_metop-c.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/iasi_metop-c.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/radiance_iasi_metop-c.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/seviri_m08.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/radiance_seviri_m08.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/seviri_m08.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/radiance_seviri_m08.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/seviri_m11.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/radiance_seviri_m11.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/seviri_m11.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/radiance_seviri_m11.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/radiance_ssmis_f17.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/ssmis_f17.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/radiance_ssmis_f17.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/radiance_ssmis_f18.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/ssmis_f18.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/radiance_ssmis_f18.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/retrieval_amv_abi_goes-16.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-16.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/retrieval_amv_abi_goes-16.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-17.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/retrieval_amv_abi_goes-17.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-17.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/retrieval_amv_abi_goes-17.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-18.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/retrieval_amv_abi_goes-18.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/satwnd.abi_goes-18.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/retrieval_amv_abi_goes-18.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h8.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/retrieval_amv_ahi_h8.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h8.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/retrieval_amv_ahi_h8.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h9.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/retrieval_amv_ahi_h9.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/satwnd.ahi_h9.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/retrieval_amv_ahi_h9.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-a.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/retrieval_amv_avhrr_metop-a.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-a.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/retrieval_amv_avhrr_metop-a.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/retrieval_amv_avhrr_metop-b.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-b.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/retrieval_amv_avhrr_metop-b.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-c.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/retrieval_amv_avhrr_metop-c.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_metop-c.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/retrieval_amv_avhrr_metop-c.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n15.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/retrieval_amv_avhrr_n15.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n15.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/retrieval_amv_avhrr_n15.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n18.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/retrieval_amv_avhrr_n18.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n18.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/retrieval_amv_avhrr_n18.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n19.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/retrieval_amv_avhrr_n19.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/satwnd.avhrr_n19.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/retrieval_amv_avhrr_n19.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.leogeo_multi.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/retrieval_amv_leogeo_multi.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/satwnd.leogeo_multi.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/retrieval_amv_leogeo_multi.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.modis_aqua.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/retrieval_amv_modis_aqua.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/satwnd.modis_aqua.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/retrieval_amv_modis_aqua.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.modis_terra.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/retrieval_amv_modis_terra.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/satwnd.modis_terra.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/retrieval_amv_modis_terra.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m10.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/retrieval_amv_seviri_m10.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m10.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/retrieval_amv_seviri_m10.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m11.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/retrieval_amv_seviri_m11.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m11.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/retrieval_amv_seviri_m11.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m8.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/retrieval_amv_seviri_m8.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m8.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/retrieval_amv_seviri_m8.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m9.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/retrieval_amv_seviri_m9.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/satwnd.seviri_m9.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/retrieval_amv_seviri_m9.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/retrieval_amv_viirs_n20.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/satwnd.viirs_n20.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/retrieval_amv_viirs_n20.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/satwnd.viirs_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/retrieval_amv_viirs_npp.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/satwnd.viirs_npp.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/retrieval_amv_viirs_npp.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-a.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/retrieval_osw_ascat_metop-a.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-a.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/retrieval_osw_ascat_metop-a.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/retrieval_osw_ascat_metop-b.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-b.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/retrieval_osw_ascat_metop-b.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-c.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/retrieval_osw_ascat_metop-c.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/scatwnd.ascat_metop-c.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/retrieval_osw_ascat_metop-c.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/ozone.omi_aura.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/retrieval_ozone_omi_aura.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/ozone.omi_aura.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/retrieval_ozone_omi_aura.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/retrieval_ozone_ompsnp_n20.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_n20.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/retrieval_ozone_ompsnp_n20.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/retrieval_ozone_ompsnp_npp.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/ozone.ompsnp_npp.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/retrieval_ozone_ompsnp_npp.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/ozone.ompstc_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/retrieval_ozone_ompstc_n20.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/ozone.ompstc_n20.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/retrieval_ozone_ompstc_n20.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/ozone.ompstc_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/retrieval_ozone_ompstc_npp.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/ozone.ompstc_npp.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/retrieval_ozone_ompstc_npp.yaml.j2 From 70300829acc63574c95a120aa14220c9fe02263b Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Mon, 12 Jan 2026 09:29:53 -0500 Subject: [PATCH 190/199] Rename some GNSSRO YAMLs (#227) Rename some GNSS-RO YAML templates so that they are consistent with the files produced by ObsForge. I decided to change these because the names are a bit more descriptive now rather that going the other way. --- .../{gnssro_geoopt.yaml.j2 => gnssro_geooptics.yaml.j2} | 0 .../atmosphere/{gnssro_k5.yaml.j2 => gnssro_kompsat5.yaml.j2} | 0 .../atmosphere/{gnssro_piq.yaml.j2 => gnssro_planetiq.yaml.j2} | 0 .../atmosphere/{gnssro_s6.yaml.j2 => gnssro_sentinel6.yaml.j2} | 0 .../atmosphere/{gnssro_tdm.yaml.j2 => gnssro_tandemx.yaml.j2} | 0 .../atmosphere/{gnssro_tsx.yaml.j2 => gnssro_terrasarx.yaml.j2} | 0 6 files changed, 0 insertions(+), 0 deletions(-) rename parm/jcb-gdas/observations/atmosphere/{gnssro_geoopt.yaml.j2 => gnssro_geooptics.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{gnssro_k5.yaml.j2 => gnssro_kompsat5.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{gnssro_piq.yaml.j2 => gnssro_planetiq.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{gnssro_s6.yaml.j2 => gnssro_sentinel6.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{gnssro_tdm.yaml.j2 => gnssro_tandemx.yaml.j2} (100%) rename parm/jcb-gdas/observations/atmosphere/{gnssro_tsx.yaml.j2 => gnssro_terrasarx.yaml.j2} (100%) diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_geoopt.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_geooptics.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/gnssro_geoopt.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/gnssro_geooptics.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_k5.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_kompsat5.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/gnssro_k5.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/gnssro_kompsat5.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_piq.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_planetiq.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/gnssro_piq.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/gnssro_planetiq.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_s6.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_sentinel6.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/gnssro_s6.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/gnssro_sentinel6.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_tdm.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_tandemx.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/gnssro_tdm.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/gnssro_tandemx.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/gnssro_tsx.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/gnssro_terrasarx.yaml.j2 similarity index 100% rename from parm/jcb-gdas/observations/atmosphere/gnssro_tsx.yaml.j2 rename to parm/jcb-gdas/observations/atmosphere/gnssro_terrasarx.yaml.j2 From 90350199f42fe69dbd02a79c600d9b300348c358 Mon Sep 17 00:00:00 2001 From: Cory Martin Date: Mon, 12 Jan 2026 12:47:00 -0500 Subject: [PATCH 191/199] Make spectralRadiance vs brightnessTemperature an option + some small conventional ob fixes (#230) Add an option (default is brightnessTemperature) to use either brightnessTemperature or spectralRadiance for CrIS and IASI. Also some minor fixes for sfc, sfcship, and sondes, as well as a simple satwind YAML to use GSI ncdiag obs. --- .../atmosphere/radiance_cris-fsr_n20.yaml.j2 | 4 +- .../atmosphere/radiance_cris-fsr_n21.yaml.j2 | 4 +- .../atmosphere/radiance_cris-fsr_npp.yaml.j2 | 4 +- .../atmosphere/radiance_iasi_metop-a.yaml.j2 | 4 +- .../atmosphere/radiance_iasi_metop-b.yaml.j2 | 4 +- .../atmosphere/radiance_iasi_metop-c.yaml.j2 | 4 +- .../observations/atmosphere/satwind.yaml.j2 | 46 +++++++++++++++++++ .../observations/atmosphere/sfc.yaml.j2 | 5 +- .../observations/atmosphere/sfcship.yaml.j2 | 2 +- .../observations/atmosphere/sondes.yaml.j2 | 4 +- 10 files changed, 72 insertions(+), 9 deletions(-) create mode 100644 parm/jcb-gdas/observations/atmosphere/satwind.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/radiance_cris-fsr_n20.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/radiance_cris-fsr_n20.yaml.j2 index e590b8bf5..d6698478c 100644 --- a/parm/jcb-gdas/observations/atmosphere/radiance_cris-fsr_n20.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/radiance_cris-fsr_n20.yaml.j2 @@ -15,9 +15,11 @@ obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 +{% if rad_to_tb | default(false) %} observed variables: [spectralRadiance] - simulated variables: [brightnessTemperature] derived variables: [brightnessTemperature] +{% endif %} + simulated variables: [brightnessTemperature] channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} # Observation Operator diff --git a/parm/jcb-gdas/observations/atmosphere/radiance_cris-fsr_n21.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/radiance_cris-fsr_n21.yaml.j2 index d6e5fa189..ad797168d 100644 --- a/parm/jcb-gdas/observations/atmosphere/radiance_cris-fsr_n21.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/radiance_cris-fsr_n21.yaml.j2 @@ -15,9 +15,11 @@ obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 +{% if rad_to_tb | default(false) %} observed variables: [spectralRadiance] - simulated variables: [brightnessTemperature] derived variables: [brightnessTemperature] +{% endif %} + simulated variables: [brightnessTemperature] channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} # Observation Operator diff --git a/parm/jcb-gdas/observations/atmosphere/radiance_cris-fsr_npp.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/radiance_cris-fsr_npp.yaml.j2 index 486d231a0..dd340d219 100644 --- a/parm/jcb-gdas/observations/atmosphere/radiance_cris-fsr_npp.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/radiance_cris-fsr_npp.yaml.j2 @@ -15,9 +15,11 @@ obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 +{% if rad_to_tb | default(false) %} observed variables: [spectralRadiance] - simulated variables: [brightnessTemperature] derived variables: [brightnessTemperature] +{% endif %} + simulated variables: [brightnessTemperature] channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} # Observation Operator diff --git a/parm/jcb-gdas/observations/atmosphere/radiance_iasi_metop-a.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/radiance_iasi_metop-a.yaml.j2 index 08e14f433..21b7f984c 100644 --- a/parm/jcb-gdas/observations/atmosphere/radiance_iasi_metop-a.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/radiance_iasi_metop-a.yaml.j2 @@ -15,9 +15,11 @@ obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 +{% if rad_to_tb | default(false) %} observed variables: [spectralRadiance] - simulated variables: [brightnessTemperature] derived variables: [brightnessTemperature] +{% endif %} + simulated variables: [brightnessTemperature] channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} # Observation Operator diff --git a/parm/jcb-gdas/observations/atmosphere/radiance_iasi_metop-b.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/radiance_iasi_metop-b.yaml.j2 index 49f48c546..8719b610a 100644 --- a/parm/jcb-gdas/observations/atmosphere/radiance_iasi_metop-b.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/radiance_iasi_metop-b.yaml.j2 @@ -15,9 +15,11 @@ obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 +{% if rad_to_tb | default(false) %} observed variables: [spectralRadiance] - simulated variables: [brightnessTemperature] derived variables: [brightnessTemperature] +{% endif %} + simulated variables: [brightnessTemperature] channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} # Observation Operator diff --git a/parm/jcb-gdas/observations/atmosphere/radiance_iasi_metop-c.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/radiance_iasi_metop-c.yaml.j2 index 2dab46e68..2aeb580b6 100644 --- a/parm/jcb-gdas/observations/atmosphere/radiance_iasi_metop-c.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/radiance_iasi_metop-c.yaml.j2 @@ -15,9 +15,11 @@ obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 +{% if rad_to_tb | default(false) %} observed variables: [spectralRadiance] - simulated variables: [brightnessTemperature] derived variables: [brightnessTemperature] +{% endif %} + simulated variables: [brightnessTemperature] channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} # Observation Operator diff --git a/parm/jcb-gdas/observations/atmosphere/satwind.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/satwind.yaml.j2 new file mode 100644 index 000000000..de5580710 --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/satwind.yaml.j2 @@ -0,0 +1,46 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: satwind + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [windEastward, windNorthward] + + # Observation Operator + # -------------------- + obs operator: + name: VertInterp + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: VertInterp + + # Observation Prior Filters (QC) + # ------------------------------ + obs prior filters: + # Apply variable changes needed for wind scaling + # For wind observations with pressure provided + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: false + + - filter: PreQC + maxvalue: 0.0 + minvalue: 0.0 + inputQC: GsiEffectiveQCGes + action: + name: reject \ No newline at end of file diff --git a/parm/jcb-gdas/observations/atmosphere/sfc.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/sfc.yaml.j2 index d6c4775ad..71ab0308d 100644 --- a/parm/jcb-gdas/observations/atmosphere/sfc.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/sfc.yaml.j2 @@ -21,8 +21,11 @@ # -------------------- obs operator: name: SfcCorrected + variables: + - name: stationPressure correction scheme to use: GSL - geovar_sfc_geomz: surface_geometric_height + station_altitude: height + geovar_sfc_geomz: height_above_mean_sea_level_at_surface geovar_geomz: geopotential_height # Linear Observation Operator diff --git a/parm/jcb-gdas/observations/atmosphere/sfcship.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/sfcship.yaml.j2 index 60d0b19bc..c0bc9e8c7 100644 --- a/parm/jcb-gdas/observations/atmosphere/sfcship.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/sfcship.yaml.j2 @@ -31,7 +31,7 @@ variables: - name: stationPressure correction scheme to use: GSL - geovar_sfc_geomz: surface_geometric_height + geovar_sfc_geomz: height_above_mean_sea_level_at_surface geovar_geomz: geopotential_height # Linear Observation Operator diff --git a/parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 index e708798fc..62cc621b8 100644 --- a/parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 +++ b/parm/jcb-gdas/observations/atmosphere/sondes.yaml.j2 @@ -19,7 +19,7 @@ obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" io pool: max pool size: 1 - simulated variables: [stationPressure, airTemperature, windEastward, windNorthward, + simulated variables: [stationPressure, airTemperature, virtualTemperature, windEastward, windNorthward, specificHumidity] # Observation Operator @@ -30,6 +30,7 @@ - name: VertInterp variables: - name: airTemperature + - name: virtualTemperature - name: windEastward - name: windNorthward - name: specificHumidity @@ -48,6 +49,7 @@ - name: VertInterp variables: - name: airTemperature + - name: virtualTemperature - name: windEastward - name: windNorthward - name: specificHumidity From e22483fc85904e318a1642f4312819b4a28282dc Mon Sep 17 00:00:00 2001 From: NicholasEsposito-NOAA <62616739+nicholasesposito@users.noreply.github.com> Date: Mon, 12 Jan 2026 12:52:58 -0500 Subject: [PATCH 192/199] remove aircraft individual variables, add merged yaml.j2 (#229) This PR merges the three individual variable aircraft yamls so the one merged file works with workflow. This fixes issue #228 and should be merged *after* jcb-gdas [PR#228](https://github.com/NOAA-EMC/jcb-gdas/issues/228) and [UFO#3951](https://github.com/JCSDA-internal/ufo/pull/3951) --------- Co-authored-by: Nicholas Esposito Co-authored-by: Cory Martin --- .../observations/atmosphere/aircraft.yaml.j2 | 767 ++++++++++++++++++ .../atmosphere/aircraft_humidity.yaml.j2 | 181 ----- .../atmosphere/aircraft_temperature.yaml.j2 | 299 ------- .../atmosphere/aircraft_wind.yaml.j2 | 290 ------- 4 files changed, 767 insertions(+), 770 deletions(-) create mode 100644 parm/jcb-gdas/observations/atmosphere/aircraft.yaml.j2 delete mode 100644 parm/jcb-gdas/observations/atmosphere/aircraft_humidity.yaml.j2 delete mode 100644 parm/jcb-gdas/observations/atmosphere/aircraft_temperature.yaml.j2 delete mode 100644 parm/jcb-gdas/observations/atmosphere/aircraft_wind.yaml.j2 diff --git a/parm/jcb-gdas/observations/atmosphere/aircraft.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/aircraft.yaml.j2 new file mode 100644 index 000000000..92334a3dd --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/aircraft.yaml.j2 @@ -0,0 +1,767 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + obsgrouping: + group variables: [stationIdentification] + sort variable: pressure + sort order: descending + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [specificHumidity, windEastward, windNorthward, airTemperature] + + # Observation Operator + # -------------------- + obs operator: + name: Composite + components: + - name: VertInterp + variables: + - name: specificHumidity + - name: airTemperature + - name: VertInterp + variables: + - name: windEastward + - name: windNorthward + # Hofx scaling + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + # Linear Observation Operator + # --------------------------- + linear obs operator: + name: Composite + components: + - name: VertInterp + variables: + - name: specificHumidity + - name: airTemperature + - name: VertInterp + variables: + - name: windEastward + - name: windNorthward + + # Hofx scaling + hofx scaling field: SurfaceWindScalingPressure + hofx scaling field group: DerivedVariables + + # Observation Bias Correction (VarBC) + # ----------------------------------- + obs bias: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_acft_suffix}}" + variables without bc: [specificHumidity, windEastward, windNorthward] + bc by record: true + variational bc: + predictors: + - name: constant + - name: obsMetadataPredictor + variable: instantaneousAltitudeRate + - name: obsMetadataPredictor + variable: instantaneousAltitudeRate + order: 2 + covariance: + minimal required obs number: 20 + variance range: [1.0e-6, 10.0] + step size: 1.0e-4 + largest analysis variance: 10000.0 + prior: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_acft_suffix}}" + inflation: + ratio: 1.1 + ratio for small dataset: 2.0 + + + obs pre filters: + + # Assign the initial observation error, based on pressure (for AIREP/ACARS; itype=130) + # (for AMDAR itype = 131, and for Canadian AMDAR itype=135) + - filter: Perform Action + filter variables: + - name: specificHumidity + where: + - variable: + name: ObsType/specificHumidity + is_in: 130, 131, 135 + action: + name: reject + #defer to post: true + + # Assign the initial observation error, based on pressure (for RECON aircraft; itype=132) + - filter: Perform Action + filter variables: + - name: specificHumidity + action: + name: assign error + error parameter: 0.2 + where: + - variable: + name: ObsType/specificHumidity + is_in: 132 + + + # Assign the initial observation error, based on pressure (for MDCRS; itype=133) + - filter: Perform Action + filter variables: + - name: specificHumidity + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000, 105000, 100000, 95000, 90000, 85000, 80000, 75000, 70000, 65000, 60000, 55000, 50000, 45000, 40000, 35000, 30000, 25000, 20000, 15000, 10000, 7500, 5000, 4000, 3000] + errors: [.19455, .19062, .18488, .17877, .17342, .16976, .16777, .16696, .16605, .16522, .16637, .17086, .17791, .18492, .18996, .19294, .19447, .19597, .19748, .19866, .19941, .19979, .19994, .19999, .20] + where: + - variable: + name: ObsType/specificHumidity + is_in: 133 + + # Assign the initial observation error, based on pressure (for TAMDAR itype=134) + - filter: Perform Action + filter variables: + - name: specificHumidity + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000, 75000] + errors: [0.2, 1000000000.0 ] + where: + - variable: + name: ObsType/specificHumidity + is_in: 134 + + # Set observation quality-realted variables + # Create PreQC group variable (pqm in GSI read_prepbufr) + - filter: Variable Assignment + assignments: + - name: PreQC/specificHumidity + type: int + source variable: QualityMarker/specificHumidity + + - filter: Variable Assignment + assignments: + - name: PreUseFlag/specificHumidity + type: int + value: 0 + + - filter: Variable Assignment + assignments: + - name: PreUseFlag/specificHumidity + type: int + value: 101 + where: + - variable: PreQC/specificHumidity + is_in: 8-15 + + - filter: Variable Assignment + assignments: + - name: PreUseFlag/specificHumidity + type: int + value: 100 + where: + - variable: ObsType/specificHumidity + is_in: 130,131,134,135 + + # Inflate if QC == 3 or 7 + - filter: Perform Action + filter variables: + - name: specificHumidity + action: + name: inflate error + inflation factor: 1.2 + where: + - variable: PreQC/specificHumidity + is_in: 3,7 + + # Assign intial ObsError specific to AIREP/ACARS + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error parameter: 3.6 + where: + - variable: + name: ObsType/windEastward + is_in: 230 + + # Assign intial ObsError specific to AMDAR + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error parameter: 3.0 + where: + - variable: + name: ObsType/windEastward + is_in: 231 + + # Assign intial ObsError specific to MDCRS + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error parameter: 2.5 + where: + - variable: + name: ObsType/windEastward + is_in: 233 + + # Assign intial ObsError specific to TAMDAR, Canadian AMDAR + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error parameter: 3.0 + where: + - variable: + name: ObsType/windEastward + is_in: 234, 235 + + # Assign the initial ObsError, based on height/pressure for RECON aircraft + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000, 70000, 65000, 60000, 55000, 50000, 45000, 40000, 35000, 30000, 25000, 20000, 15000, 10000, 7500, 5000] + errors: [2.4, 2.4, 2.5, 2.6, 2.7, 2.8, 2.95, 3.1, 3.25, 3.4, 3.175, 2.95, 2.725, 2.5, 2.6, 2.7] + where: + - variable: + name: ObsType/windEastward + is_in: 232 + + # Set observation quality-realted variables + # Create PreQC group variable (pqm in GSI read_prepbufr) + - filter: Variable Assignment + assignments: + - name: PreQC/windNorthward + type: int + source variable: QualityMarker/windNorthward + + # Set observation quality-realted variables + # Create PreQC group variable (pqm in GSI read_prepbufr) + - filter: Variable Assignment + assignments: + - name: PreQC/windEastward + type: int + source variable: QualityMarker/windEastward + + - filter: Variable Assignment + assignments: + - name: PreUseFlag/windNorthward + type: int + value: 0 + + - filter: Variable Assignment + assignments: + - name: PreUseFlag/windEastward + type: int + value: 0 + + - filter: Variable Assignment + assignments: + - name: PreUseFlag/windNorthward + type: int + value: 101 + where: + - variable: PreQC/windNorthward + is_in: 8-15 + + - filter: Variable Assignment + assignments: + - name: PreUseFlag/windEastward + type: int + value: 101 + where: + - variable: PreQC/windEastward + is_in: 8-15 + + # Error inflation when QC is 3 or 7 + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: inflate error + inflation factor: 1.2 + where: + - variable: PreQC/windNorthward + is_in: 3,7 + + # Error inflation when observation pressure < 50 hPa (read_prepbufr.f90) + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + action: + name: inflate error + inflation factor: 1.2 + where: + - variable: MetaData/pressure + maxvalue: 5000.0 + + # Error assignments + # Assign the initial observation error, based on pressure (for AIREP/ACARS; itype=130) + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [100000, 95000, 90000, 85000, 80000] + errors: [2.5, 2.3, 2.1, 1.9, 1.7] + where: + - variable: + name: ObsType/airTemperature + is_in: 130 + + # Assign the initial observation error, based on pressure (for AMDAR and MDCRS; itype=131,133) + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [100000, 95000, 90000, 85000, 80000] + errors: [1.4706, 1.3529, 1.2353, 1.1176, 1.0] + where: + - variable: + name: ObsType/airTemperature + is_in: 131,133 + + # Assign the initial observation error, based on pressure (for RECON aircraft; itype=132) + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [110000, 100000, 95000, 90000, 85000, 35000, 30000, 25000, 20000, 15000, 10000, 7500, 5000, 4000, 3000, 2000, 1000] + errors: [ 1.2, 1.2, 1.1, 0.9, 0.8, 0.8, 0.9, 1.2, 1.2, 1.0, 0.8, 0.8, 0.9, 0.95, 1.0, 1.25, 1.5] + where: + - variable: + name: ObsType/airTemperature + is_in: 132 + + # Assign the initial observation error, based on pressure (for TAMDAR itype=134) + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [100000, 95000, 90000, 85000, 80000, 60000, 40000, 7500] + errors: [1.5, 1.35, 1.25, 1.10, 1.0, 1.3, 1.7, 1000000000.0 ] + where: + - variable: + name: ObsType/airTemperature + is_in: 134 + + # Assign the initial observation error, based on pressure (for Canadian AMDAR itype=135) + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelStepwiseLinear + options: + xvar: + name: MetaData/pressure + xvals: [100000, 95000, 90000, 85000, 80000] + errors: [1.4706, 1.3529, 1.2353, 1.1176, 1.0 ] + where: + - variable: + name: ObsType/airTemperature + is_in: 135 + + # Set observation quality-realted variables + # Create PreQC group variable (pqm in GSI read_prepbufr) + - filter: Variable Assignment + assignments: + - name: PreQC/airTemperature + type: int + source variable: QualityMarker/airTemperature + + - filter: Variable Assignment + assignments: + - name: PreUseFlag/airTemperature + type: int + value: 0 + + - filter: Variable Assignment + assignments: + - name: PreUseFlag/airTemperature + type: int + value: 101 + where: + - variable: PreQC/airTemperature + is_in: 8-15 + + + # Reassign any AIREP/PIREP (type=130) airTemperature MetaData/stationIdentification to 'KX130 ' + - filter: Variable Assignment + where: + - variable: + name: ObsType/airTemperature + is_in: 130 + assignments: + - name: MetaData/stationIdentification + value: "KX130" + + # Error inflation where QC is 3 or 7 + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: inflate error + inflation factor: 1.2 + where: + - variable: PreQC/airTemperature + is_in: 3,7 + + # Error inflation when observation pressure < 100 mb (read_prepbufr.f90) + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: inflate error + inflation factor: 1.2 + where: + - variable: MetaData/pressure + maxvalue: 10000.0 + + # Inflate error by 10 if kx==130 and P >=500 mb (read_prepbufr.f90) + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: inflate error + inflation factor: 10.0 + where: + - variable: ObsType/airTemperature + is_in: 130.0 + - variable: MetaData/pressure + minvalue: 50000.0 + + # Inflate if IALR > 30 + - filter: Perform Action + filter variables: + - name: airTemperature + action: + name: inflate error + inflation factor: 1.5 + where: + - variable: MetaData/instantaneousAltitudeRate + minvalue: 30.0 + + + obs prior filters: + + # Calculates ratio_errors from GSI + # Error inflation based on pressure check + - filter: Perform Action + filter variables: + - name: specificHumidity + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: specificHumidity + inflation factor: 8.0 + request_saturation_specific_humidity_geovals: true + geovar_sfc_geomz: height_above_mean_sea_level_at_surface + + + # Apply variable changes needed for performing wind scaling + # --------------------------------------------------------- + - filter: Variable Transforms + Transform: SurfaceWindScalingPressure + SkipWhenNoObs: True + + + # Error inflation based on pressure check + - filter: Perform Action + filter variables: + - name: windEastward + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windEastward + inflation factor: 4.0 + surface_obs: false + geovar_sfc_geomz: height_above_mean_sea_level_at_surface + defer to post: true + + # Error inflation based on pressure check + - filter: Perform Action + filter variables: + - name: windNorthward + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: windNorthward + inflation factor: 4.0 + surface_obs: false + #test_obserr: GsiAdjustObsError + geovar_sfc_geomz: height_above_mean_sea_level_at_surface + defer to post: true + + + # error inflation based on pressure check + - filter: Perform Action + filter variables: + - name: airTemperature + where: + - variable: + name: ObsType/airTemperature + is_in: 130-135 + action: + name: inflate error + inflation variable: + name: ObsFunction/ObsErrorFactorPressureCheck + options: + variable: airTemperature + inflation factor: 8.0 + surface_obs: true + geovar_sfc_geomz: height_above_mean_sea_level_at_surface + defer to post: true + + + obs post filters: + + # Background check + # ratio threshold 8.0 + - filter: Background Check + test_hofx: HofX + filter variables: + - name: specificHumidity + threshold: 8.0 + action: + name: reject + where: + - variable: PreQC/specificHumidity + is_not_in: 3,7 + defer to post: true + + # ratio threshold * 0.7 if PreQC=3 + - filter: Background Check + test_hofx: HofX + filter variables: + - name: specificHumidity + threshold: 5.6 + action: + name: reject + where: + - variable: PreQC/specificHumidity + is_in: 3,7 + defer to post: true + + + # Reject where QC > 8 + - filter: Perform Action + filter variables: + - name: specificHumidity + where: + - variable: PreQC/specificHumidity + is_in: 9-15 + action: + name: reject + + # Gross Check + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 230, 231, 232, 233, 234, 235 ] + cgross: [ 6.0, 6.5, 7.0, 7.5, 7.5, 7.5 ] + error_min: [ 1.4, 1.4, 1.4, 1.4, 1.4, 1.4 ] + error_max: [ 6.1, 6.1, 6.1, 6.1, 6.1, 6.1 ] + variable: windEastward + where: + - variable: PreQC/windEastward + is_not_in: 3 + action: + name: reject + defer to post: true + + # Gross Check + - filter: Background Check + filter variables: + - name: windEastward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 230, 231, 232, 233, 234, 235 ] + cgross: [ 4.2, 4.55, 4.9, 5.25, 5.25, 5.25 ] + error_min: [ 1.4, 1.4, 1.4, 1.4, 1.4, 1.4 ] + error_max: [ 6.1, 6.1, 6.1, 6.1, 6.1, 6.1 ] + variable: windEastward + where: + - variable: PreQC/windEastward + is_in: 3 + action: + name: reject + defer to post: true + + # Gross Check + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 230, 231, 232, 233, 234, 235 ] + cgross: [ 6.0, 6.5, 7.0, 7.5, 7.5, 7.5 ] + error_min: [ 1.4, 1.4, 1.4, 1.4, 1.4, 1.4 ] + error_max: [ 6.1, 6.1, 6.1, 6.1, 6.1, 6.1 ] + variable: windNorthward + where: + - variable: PreQC/windNorthward + is_not_in: 3 + action: + name: reject + defer to post: true + + # Gross Check + - filter: Background Check + filter variables: + - name: windNorthward + function absolute threshold: + - name: ObsFunction/WindsSPDBCheck + options: + wndtype: [ 230, 231, 232, 233, 234, 235 ] + cgross: [ 4.2, 4.55, 4.9, 5.25, 5.25, 5.25 ] + error_min: [ 1.4, 1.4, 1.4, 1.4, 1.4, 1.4 ] + error_max: [ 6.1, 6.1, 6.1, 6.1, 6.1, 6.1 ] + variable: windNorthward + where: + - variable: PreQC/windNorthward + is_in: 3 + action: + name: reject + defer to post: true + + + # Reject where QC is 9-15 + - filter: Perform Action + filter variables: + - name: windEastward + - name: windNorthward + where: + - variable: PreQC/windNorthward + is_in: 9-15 + action: + name: reject + defer to post: true + + - filter: Variable Assignment + assignments: + - name: HofXBc/airTemperature + type: float + function: + name: ObsFunction/LinearCombination + options: + variables: [HofX/airTemperature] + coefs: [1.0] + defer to post: true + + # Background check + # ratio threshold 7.0 + - filter: Background Check + test_hofx: HofXBc + filter variables: + - name: airTemperature + threshold: 7.0 + action: + name: reject + where: + - variable: PreQC/airTemperature + is_not_in: [3] + defer to post: true + + # ratio threshold * 0.7 if PreQC=3 + - filter: Background Check + test_hofx: HofXBc + filter variables: + - name: airTemperature + threshold: 4.9 + action: + name: reject + where: + - variable: PreQC/airTemperature + is_in: [3] + defer to post: true + + # Reject where QC = 4-19 + - filter: Perform Action + filter variables: + - name: airTemperature + where: + - variable: PreQC/airTemperature + is_in: 4-19 + action: + name: reject + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 8061 #specificHumidity + #passedBenchmark: 227312 #windNorthward and windEastward + #passedBenchmark: 114568 #airTemperature diff --git a/parm/jcb-gdas/observations/atmosphere/aircraft_humidity.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/aircraft_humidity.yaml.j2 deleted file mode 100644 index 07807fe0e..000000000 --- a/parm/jcb-gdas/observations/atmosphere/aircraft_humidity.yaml.j2 +++ /dev/null @@ -1,181 +0,0 @@ -- - - # Observation Space (I/O) - # ----------------------- - obs space: - name: {{observation_from_jcb}} - obsdatain: - engine: - type: H5File - obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" - missing file action: warn - obsgrouping: - group variables: [stationIdentification] - sort variable: pressure - sort order: descending - obsdataout: - engine: - type: H5File - obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" - io pool: - max pool size: 1 - simulated variables: [specificHumidity] - - # Observation Operator - # -------------------- - obs operator: - name: Composite - components: - - name: VertInterp - variables: [specificHumidity ] - - # Linear Observation Operator - # --------------------------- - linear obs operator: - name: Composite - components: - - name: VertInterp - variables: [specificHumidity] - - obs pre filters: - - # Assign the initial observation error, based on pressure (for AIREP/ACARS; itype=130) - # (for AMDAR itype = 131, and for Canadian AMDAR itype=135) - - filter: Perform Action - filter variables: - - name: specificHumidity - where: - - variable: - name: ObsType/specificHumidity - is_in: 130, 131, 135 - action: - name: reject - #defer to post: true - - # Assign the initial observation error, based on pressure (for RECON aircraft; itype=132) - - filter: Perform Action - filter variables: - - name: specificHumidity - action: - name: assign error - error parameter: 0.2 - where: - - variable: - name: ObsType/specificHumidity - is_in: 132 - - - # Assign the initial observation error, based on pressure (for MDCRS; itype=133) - - filter: Perform Action - filter variables: - - name: specificHumidity - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [110000, 105000, 100000, 95000, 90000, 85000, 80000, 75000, 70000, 65000, 60000, 55000, 50000, 45000, 40000, 35000, 30000, 25000, 20000, 15000, 10000, 7500, 5000, 4000, 3000] - errors: [.19455, .19062, .18488, .17877, .17342, .16976, .16777, .16696, .16605, .16522, .16637, .17086, .17791, .18492, .18996, .19294, .19447, .19597, .19748, .19866, .19941, .19979, .19994, .19999, .20] - where: - - variable: - name: ObsType/specificHumidity - is_in: 133 - - # Assign the initial observation error, based on pressure (for TAMDAR itype=134) - - filter: Perform Action - filter variables: - - name: specificHumidity - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [110000, 75000] - errors: [0.2, 1000000000.0 ] - where: - - variable: - name: ObsType/specificHumidity - is_in: 134 - - # Inflate if QC == 3 or 7 - - filter: Perform Action - filter variables: - - name: specificHumidity - action: - name: inflate error - inflation factor: 1.2 - where: - - variable: PreQC/specificHumidity - is_in: 3,7 - - obs prior filters: - - # Calculates ratio_errors from GSI - # Error inflation based on pressure check - - filter: Perform Action - filter variables: - - name: specificHumidity - action: - name: inflate error - inflation variable: - name: ObsFunction/ObsErrorFactorPressureCheck - options: - variable: specificHumidity - inflation factor: 8.0 - request_saturation_specific_humidity_geovals: true - geovar_sfc_geomz: height_above_mean_sea_level_at_surface - - - obs post filters: - - # Background check - # ratio threshold 8.0 - - filter: Background Check - test_hofx: HofX - filter variables: - - name: specificHumidity - threshold: 8.0 - action: - name: reject - where: - - variable: PreQC/specificHumidity - is_not_in: 3,7 - defer to post: true - - # ratio threshold * 0.7 if PreQC=3 - - filter: Background Check - test_hofx: HofX - filter variables: - - name: specificHumidity - threshold: 5.6 - action: - name: reject - where: - - variable: PreQC/specificHumidity - is_in: 3,7 - defer to post: true - - - # Reject where QC > 8 - - filter: Perform Action - filter variables: - - name: specificHumidity - where: - - variable: PreQC/specificHumidity - is_in: 9-15 - action: - name: reject - - - # GeoVaLs for Driving Observation Operators (testing mode) - # -------------------------------------------------------- - geovals: - filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" - - # Passed benchmark for UFO testing - # -------------------------------- - passedBenchmark: 8061 diff --git a/parm/jcb-gdas/observations/atmosphere/aircraft_temperature.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/aircraft_temperature.yaml.j2 deleted file mode 100644 index 8c6e03d6b..000000000 --- a/parm/jcb-gdas/observations/atmosphere/aircraft_temperature.yaml.j2 +++ /dev/null @@ -1,299 +0,0 @@ -- - - # Observation Space (I/O) - # ----------------------- - obs space: - name: {{observation_from_jcb}} - obsdatain: - engine: - type: H5File - obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" - missing file action: warn - obsgrouping: - group variables: [stationIdentification] - sort variable: pressure - sort order: descending - obsdataout: - engine: - type: H5File - obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" - io pool: - max pool size: 1 - simulated variables: [airTemperature] - - # Observation Operator - # -------------------- - obs operator: - name: Composite - components: - - name: VertInterp - variables: [airTemperature] - - # Linear Observation Operator - # --------------------------- - linear obs operator: - name: Composite - components: - - name: VertInterp - variables: [airTemperature] - - # Observation Bias Correction (VarBC) - # ----------------------------------- - obs bias: - input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_acft_suffix}}" - output file: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_acft_suffix}}" - bc by record: true - variational bc: - predictors: - - name: constant - - name: obsMetadataPredictor - variable: instantaneousAltitudeRate - - name: obsMetadataPredictor - variable: instantaneousAltitudeRate - order: 2 - covariance: - minimal required obs number: 3 - variance range: [1.0e-6, 1.0] - step size: 1.0e-4 - largest analysis variance: 10000.0 - prior: - input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiascovin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovin_acft_suffix}}" - inflation: - ratio: 1.005 - ratio for small dataset: 2.0 - output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_acft_suffix}}" - - - obs pre filters: - - # Error assignments - # Assign the initial observation error, based on pressure (for AIREP/ACARS; itype=130) - - filter: Perform Action - filter variables: - - name: airTemperature - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [100000, 95000, 90000, 85000, 80000] - errors: [2.5, 2.3, 2.1, 1.9, 1.7] - where: - - variable: - name: ObsType/airTemperature - is_in: 130 - - # Assign the initial observation error, based on pressure (for AMDAR and MDCRS; itype=131,133) - - filter: Perform Action - filter variables: - - name: airTemperature - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [100000, 95000, 90000, 85000, 80000] - errors: [1.4706, 1.3529, 1.2353, 1.1176, 1.0] - where: - - variable: - name: ObsType/airTemperature - is_in: 131,133 - - # Assign the initial observation error, based on pressure (for RECON aircraft; itype=132) - - filter: Perform Action - filter variables: - - name: airTemperature - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [110000, 100000, 95000, 90000, 85000, 35000, 30000, 25000, 20000, 15000, 10000, 7500, 5000, 4000, 3000, 2000, 1000] - errors: [ 1.2, 1.2, 1.1, 0.9, 0.8, 0.8, 0.9, 1.2, 1.2, 1.0, 0.8, 0.8, 0.9, 0.95, 1.0, 1.25, 1.5] - where: - - variable: - name: ObsType/airTemperature - is_in: 132 - - # Assign the initial observation error, based on pressure (for TAMDAR itype=134) - - filter: Perform Action - filter variables: - - name: airTemperature - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [100000, 95000, 90000, 85000, 80000, 60000, 40000, 7500] - errors: [1.5, 1.35, 1.25, 1.10, 1.0, 1.3, 1.7, 1000000000.0 ] - where: - - variable: - name: ObsType/airTemperature - is_in: 134 - - # Assign the initial observation error, based on pressure (for Canadian AMDAR itype=135) - - filter: Perform Action - filter variables: - - name: airTemperature - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [100000, 95000, 90000, 85000, 80000] - errors: [1.4706, 1.3529, 1.2353, 1.1176, 1.0 ] - where: - - variable: - name: ObsType/airTemperature - is_in: 135 - -# Future section to re-assign kx0 here - - # Reassign any AIREP/PIREP (type=130) airTemperature MetaData/stationIdentification to 'KX130 ' - - filter: Variable Assignment - where: - - variable: - name: ObsType/airTemperature - is_in: 130 - assignments: - - name: MetaData/stationIdentification - value: "KX130" - - # Error inflation where QC is 3 or 7 - - filter: Perform Action - filter variables: - - name: airTemperature - action: - name: inflate error - inflation factor: 1.2 - where: - - variable: PreQC/airTemperature - is_in: 3,7 - - # Error inflation when observation pressure < 100 mb (read_prepbufr.f90) - - filter: Perform Action - filter variables: - - name: airTemperature - action: - name: inflate error - inflation factor: 1.2 - where: - - variable: MetaData/pressure - maxvalue: 10000.0 - - # Inflate error by 10 if kx==130 and P >=500 mb (read_prepbufr.f90) - - filter: Perform Action - filter variables: - - name: airTemperature - action: - name: inflate error - inflation factor: 10.0 - where: - - variable: ObsType/airTemperature - is_in: 130.0 - - variable: MetaData/pressure - minvalue: 50000.0 - - # Inflate if IALR > 30 - - filter: Perform Action - filter variables: - - name: airTemperature - action: - name: inflate error - inflation factor: 1.5 - where: - - variable: MetaData/instantaneousAltitudeRate - minvalue: 30.0 - - - obs prior filters: - - # error inflation based on pressure check - - filter: Perform Action - filter variables: - - name: airTemperature - where: - - variable: - name: ObsType/airTemperature - is_in: 130-135 - action: - name: inflate error - inflation variable: - name: ObsFunction/ObsErrorFactorPressureCheck - options: - variable: airTemperature - inflation factor: 8.0 - surface_obs: true - geovar_sfc_geomz: height_above_mean_sea_level_at_surface - defer to post: true - - - obs post filters: - - - filter: Variable Assignment - assignments: - - name: HofXBc/airTemperature - type: float - function: - name: ObsFunction/LinearCombination - options: - variables: [HofX/airTemperature] - coefs: [1.0] - defer to post: true - - # Background check - # ratio threshold 7.0 - - filter: Background Check - test_hofx: HofXBc - filter variables: - - name: airTemperature - threshold: 7.0 - action: - name: reject - where: - - variable: PreQC/airTemperature - is_not_in: [3] - defer to post: true - - # ratio threshold * 0.7 if PreQC=3 - - filter: Background Check - test_hofx: HofXBc - filter variables: - - name: airTemperature - threshold: 4.9 - action: - name: reject - where: - - variable: PreQC/airTemperature - is_in: [3] - defer to post: true - - # Reject where QC = 4-19 - - filter: Perform Action - filter variables: - - name: airTemperature - where: - - variable: PreQC/airTemperature - is_in: 4-19 - action: - name: reject - - - # GeoVaLs for Driving Observation Operators (testing mode) - # -------------------------------------------------------- - geovals: - filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" - - # Passed benchmark for UFO testing - # -------------------------------- - passedBenchmark: 114568 diff --git a/parm/jcb-gdas/observations/atmosphere/aircraft_wind.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/aircraft_wind.yaml.j2 deleted file mode 100644 index 03c4b78e8..000000000 --- a/parm/jcb-gdas/observations/atmosphere/aircraft_wind.yaml.j2 +++ /dev/null @@ -1,290 +0,0 @@ -- - - # Observation Space (I/O) - # ----------------------- - obs space: - name: {{observation_from_jcb}} - obsdatain: - engine: - type: H5File - obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" - missing file action: warn - obsgrouping: - group variables: [stationIdentification] - sort variable: pressure - sort order: descending - obsdataout: - engine: - type: H5File - obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" - io pool: - max pool size: 1 - simulated variables: [windEastward, windNorthward] - - # Observation Operator - # -------------------- - obs operator: - name: Composite - components: - - name: VertInterp - variables: - - name: windEastward - - name: windNorthward - # Hofx scaling - hofx scaling field: SurfaceWindScalingPressure - hofx scaling field group: DerivedVariables - - - # Linear Observation Operator - # --------------------------- - linear obs operator: - name: Composite - components: - - name: VertInterp - variables: - - name: windEastward - - name: windNorthward - - # Hofx scaling - hofx scaling field: SurfaceWindScalingPressure - hofx scaling field group: DerivedVariables - - - obs pre filters: - # Assign intial ObsError specific to AIREP/ACARS - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - action: - name: assign error - error parameter: 3.6 - where: - - variable: - name: ObsType/windEastward - is_in: 230 - - # Assign intial ObsError specific to AMDAR - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - action: - name: assign error - error parameter: 3.0 - where: - - variable: - name: ObsType/windEastward - is_in: 231 - - # Assign intial ObsError specific to MDCRS - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - action: - name: assign error - error parameter: 2.5 - where: - - variable: - name: ObsType/windEastward - is_in: 233 - - - # Assign intial ObsError specific to TAMDAR, Canadian AMDAR - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - action: - name: assign error - error parameter: 3.0 - where: - - variable: - name: ObsType/windEastward - is_in: 234, 235 - - # Assign the initial ObsError, based on height/pressure for RECON aircraft - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - action: - name: assign error - error function: - name: ObsFunction/ObsErrorModelStepwiseLinear - options: - xvar: - name: MetaData/pressure - xvals: [110000, 70000, 65000, 60000, 55000, 50000, 45000, 40000, 35000, 30000, 25000, 20000, 15000, 10000, 7500, 5000] - errors: [2.4, 2.4, 2.5, 2.6, 2.7, 2.8, 2.95, 3.1, 3.25, 3.4, 3.175, 2.95, 2.725, 2.5, 2.6, 2.7] - where: - - variable: - name: ObsType/windEastward - is_in: 232 - - # Error inflation when QC is 3 or 7 - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - action: - name: inflate error - inflation factor: 1.2 - where: - - variable: PreQC/windNorthward - is_in: 3,7 - - # Error inflation when observation pressure < 50 hPa (read_prepbufr.f90) - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - action: - name: inflate error - inflation factor: 1.2 - where: - - variable: MetaData/pressure - maxvalue: 5000.0 - - - obs prior filters: - - # Apply variable changes needed for performing wind scaling - # --------------------------------------------------------- - - filter: Variable Transforms - Transform: SurfaceWindScalingPressure - SkipWhenNoObs: True - - - # Error inflation based on pressure check - - filter: Perform Action - filter variables: - - name: windEastward - action: - name: inflate error - inflation variable: - name: ObsFunction/ObsErrorFactorPressureCheck - options: - variable: windEastward - inflation factor: 4.0 - surface_obs: false - geovar_sfc_geomz: height_above_mean_sea_level_at_surface - defer to post: true - - # Error inflation based on pressure check - - filter: Perform Action - filter variables: - - name: windNorthward - action: - name: inflate error - inflation variable: - name: ObsFunction/ObsErrorFactorPressureCheck - options: - variable: windNorthward - inflation factor: 4.0 - surface_obs: false - #test_obserr: GsiAdjustObsError - geovar_sfc_geomz: height_above_mean_sea_level_at_surface - defer to post: true - - - obs post filters: - - # Gross Check - - filter: Background Check - filter variables: - - name: windEastward - function absolute threshold: - - name: ObsFunction/WindsSPDBCheck - options: - wndtype: [ 230, 231, 232, 233, 234, 235 ] - cgross: [ 6.0, 6.5, 7.0, 7.5, 7.5, 7.5 ] - error_min: [ 1.4, 1.4, 1.4, 1.4, 1.4, 1.4 ] - error_max: [ 6.1, 6.1, 6.1, 6.1, 6.1, 6.1 ] - variable: windEastward - where: - - variable: PreQC/windEastward - is_not_in: 3 - action: - name: reject - defer to post: true - - # Gross Check - - filter: Background Check - filter variables: - - name: windEastward - function absolute threshold: - - name: ObsFunction/WindsSPDBCheck - options: - wndtype: [ 230, 231, 232, 233, 234, 235 ] - cgross: [ 4.2, 4.55, 4.9, 5.25, 5.25, 5.25 ] - error_min: [ 1.4, 1.4, 1.4, 1.4, 1.4, 1.4 ] - error_max: [ 6.1, 6.1, 6.1, 6.1, 6.1, 6.1 ] - variable: windEastward - where: - - variable: PreQC/windEastward - is_in: 3 - action: - name: reject - defer to post: true - - # Gross Check - - filter: Background Check - filter variables: - - name: windNorthward - function absolute threshold: - - name: ObsFunction/WindsSPDBCheck - options: - wndtype: [ 230, 231, 232, 233, 234, 235 ] - cgross: [ 6.0, 6.5, 7.0, 7.5, 7.5, 7.5 ] - error_min: [ 1.4, 1.4, 1.4, 1.4, 1.4, 1.4 ] - error_max: [ 6.1, 6.1, 6.1, 6.1, 6.1, 6.1 ] - variable: windNorthward - where: - - variable: PreQC/windNorthward - is_not_in: 3 - action: - name: reject - defer to post: true - - # Gross Check - - filter: Background Check - filter variables: - - name: windNorthward - function absolute threshold: - - name: ObsFunction/WindsSPDBCheck - options: - wndtype: [ 230, 231, 232, 233, 234, 235 ] - cgross: [ 4.2, 4.55, 4.9, 5.25, 5.25, 5.25 ] - error_min: [ 1.4, 1.4, 1.4, 1.4, 1.4, 1.4 ] - error_max: [ 6.1, 6.1, 6.1, 6.1, 6.1, 6.1 ] - variable: windNorthward - where: - - variable: PreQC/windNorthward - is_in: 3 - action: - name: reject - defer to post: true - - # Reject where QC is 9-15 - - filter: Perform Action - filter variables: - - name: windEastward - - name: windNorthward - where: - - variable: PreQC/windNorthward - is_in: 9-15 - action: - name: reject - defer to post: true - - - # GeoVaLs for Driving Observation Operators (testing mode) - # -------------------------------------------------------- - geovals: - filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" - - # Passed benchmark for UFO testing - # -------------------------------- - passedBenchmark: 227312 From 100264c3b400d35e233837347576db2d441f9511 Mon Sep 17 00:00:00 2001 From: Bo Huang <40745309+bhuang95@users.noreply.github.com> Date: Fri, 16 Jan 2026 11:38:48 -0700 Subject: [PATCH 193/199] Enable EnKF-only for atmosphere (#219) This PR works along with the following three dependent PRs to enable the EnKF-only configuration for the atmosphere within the global workflow (see detailed description https://github.com/NOAA-EMC/global-workflow/pull/4345) Dependencies: -https://github.com/NOAA-EMC/global-workflow/pull/4345 -https://github.com/NOAA-EMC/GDASApp/pull/2010 -https://github.com/NOAA-EMC/jcb/pull/32 Resolve - https://github.com/NOAA-EMC/global-workflow/issues/4339 --------- Co-authored-by: Cory Martin --- .../algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 index 931572742..9cf1fb727 100644 --- a/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 +++ b/parm/jcb-gdas/algorithm/atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 @@ -106,10 +106,16 @@ members from template: datapath: ./anl/mem%mem% filename: cubed_sphere_grid_atminc.jedi.nc fv3 increment output: +{% if atm_enkfonly | default(false) %} + filetype: fms restart + is restart: false + filename_nonrestart: {{ atmosphere_ensemble_analysis_prefix }}cubed_sphere_grid_atminc.nc +{% else %} filetype: cube sphere history provider: ufs - datapath: ./anl/mem%mem% filename: {{ atmosphere_ensemble_analysis_prefix }}cubed_sphere_grid_atminc.nc +{% endif %} + datapath: ./anl/mem%mem% field io names: eastward_wind: u_inc northward_wind: v_inc From 257ce2cd986719ca8022ad99b24cd4cc79a76214 Mon Sep 17 00:00:00 2001 From: "Jianjun Jin, NOAA EMC" Date: Fri, 16 Jan 2026 13:40:22 -0500 Subject: [PATCH 194/199] Configurations to assimilate GPM GMI data in all-sky. (#90) Quality control and bias correction configurations to assimilate GPM GMI data in all-sky conditions. The related issue is https://github.com/NOAA-EMC/GDASApp/issues/1537 . ## Dependencies List the other PRs that this PR is dependent on: - https://github.com/JCSDA-internal/ufo/pull/3665 - https://github.com/JCSDA-internal/ufo/pull/3673 --------- Co-authored-by: Dan Holdaway <27729500+danholdaway@users.noreply.github.com> Co-authored-by: DavidNew-NOAA <134300700+DavidNew-NOAA@users.noreply.github.com> --- .../atmosphere/radiance_gmi_gpm.yaml | 33 ++ .../atmosphere/radiance_gmi_gpm.yaml.j2 | 431 ++++++++++++++++++ 2 files changed, 464 insertions(+) create mode 100644 parm/jcb-gdas/observation_chronicle/atmosphere/radiance_gmi_gpm.yaml create mode 100644 parm/jcb-gdas/observations/atmosphere/radiance_gmi_gpm.yaml.j2 diff --git a/parm/jcb-gdas/observation_chronicle/atmosphere/radiance_gmi_gpm.yaml b/parm/jcb-gdas/observation_chronicle/atmosphere/radiance_gmi_gpm.yaml new file mode 100644 index 000000000..19ff597a4 --- /dev/null +++ b/parm/jcb-gdas/observation_chronicle/atmosphere/radiance_gmi_gpm.yaml @@ -0,0 +1,33 @@ +# Instrument metadata +# ------------------- +commissioned: 2014-04-01T00:00:00 +observer_type: satellite # Type of chronicle to use + +# Instrument initial configuration +# -------------------------------- +channel_variables: + simulated: min + active: min + biascorrtd: min + error: max + error_cld: max + error_cld1: max + ermax: max + x0: max + x1: max + x2: max +channel_values: +# sim act bcd err errcld errcl1 ermax x0 x1 x2 + 1: [ 1, 1, 1, 2.70, 14.00, 6.00, 30.00, 0.050, 0.200, 0.500] + 2: [ 1, 1, 1, 3.70, 15.00, 7.00, 30.00, 0.050, 0.200, 0.500] + 3: [ 1, 1, 1, 3.50, 10.00, 7.00, 15.00, 0.050, 0.200, 0.500] + 4: [ 1, 1, 1, 4.50, 20.00, 16.00, 30.00, 0.050, 0.200, 0.500] + 5: [ 1, 1, 1, 4.00, 9.00, 7.00, 15.00, 0.050, 0.200, 0.500] + 6: [ 1, 1, 1, 3.80, 16.00, 13.00, 30.00, 0.050, 0.200, 0.500] + 7: [ 1, 1, 1, 20.00, 40.00, 23.00, 15.00, 0.050, 0.200, 0.500] + 8: [ 1, 1, 1, 5.00, 10.00, 10.00, 15.00, 0.050, 0.200, 0.500] + 9: [ 1, 1, 1, 11.50, 13.00, 12.00, 20.00, 0.050, 0.200, 0.500] + 10: [ 1, 1, 1, 5.00, 12.00, 8.00, 20.00, 0.050, 0.300, 0.500] + 11: [ 1, 1, 1, 5.00, 12.00, 8.00, 20.00, 0.050, 0.200, 0.500] + 12: [ 1, 1, 1, 2.50, 8.00, 4.00, 10.00, 0.050, 0.300, 0.500] + 13: [ 1, 1, 1, 3.00, 8.00, 7.00, 10.00, 0.050, 0.300, 0.500] diff --git a/parm/jcb-gdas/observations/atmosphere/radiance_gmi_gpm.yaml.j2 b/parm/jcb-gdas/observations/atmosphere/radiance_gmi_gpm.yaml.j2 new file mode 100644 index 000000000..b0001fffd --- /dev/null +++ b/parm/jcb-gdas/observations/atmosphere/radiance_gmi_gpm.yaml.j2 @@ -0,0 +1,431 @@ +- + + # Observation Space (I/O) + # ----------------------- + obs space: + name: {{observation_from_jcb}} + obsdatain: + engine: + type: H5File + obsfile: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}{{atmosphere_obsdatain_suffix}}" + missing file action: warn + obsdataout: + engine: + type: H5File + obsfile: "{{atmosphere_obsdataout_path}}/{{atmosphere_obsdataout_prefix}}{{observation_from_jcb}}{{atmosphere_obsdataout_suffix}}" + io pool: + max pool size: 1 + simulated variables: [brightnessTemperature] + channels: &{{observation_from_jcb}}_simulated_channels {{ get_satellite_variable(observation_from_jcb, "simulated") }} + + # Observation Operator + # -------------------- + obs operator: + name: CRTM + Absorbers: [H2O, O3, CO2] + Clouds: &{{observation_from_jcb}}_clouds [Water, Ice, Rain, Snow, Graupel] +# Cloud_Fraction: 1.0 +# ---methods for cloud fraction and radii in fov: thompson, or none + method for cloud fraction within fov: thompson + method for hydrometeor effective radii within fov: thompson + Cloud_Seeding: true + obs options: + Sensor_ID: &{{observation_from_jcb}}_sensor_id gmi_gpm + EndianType: little_endian + CoefficientPath: "{{crtm_coefficient_path}}" + linear obs operator: + Absorbers: [H2O, O3] + + # Observation Bias Correction (VarBC) + # ----------------------------------- + obs bias: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiasin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasin_suffix}}" + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiasout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiasout_suffix}}" + variables without bc: [brightnessTemperature] + channels: {{ get_satellite_variable(observation_from_jcb, "not_biascorrtd") }} + variational bc: + predictors: + - name: constant + - name: lapseRate + order: 2 + tlapse: &{{observation_from_jcb}}_tlapse "{{atmosphere_obsbiasin_path}}/{{atmosphere_obstlapsein_prefix}}{{observation_from_jcb}}{{atmosphere_obstlapsein_suffix}}" + - name: lapseRate + tlapse: *{{observation_from_jcb}}_tlapse + - name: emissivityJacobian + - name: sensorScanAngle + var_name: sensorScanPosition + order: 4 + - name: sensorScanAngle + var_name: sensorScanPosition + order: 3 + - name: sensorScanAngle + var_name: sensorScanPosition + order: 2 + - name: sensorScanAngle + var_name: sensorScanPosition + covariance: + minimal required obs number: 20 + variance range: [1.0e-6, 10.0] + step size: 1.0e-4 + largest analysis variance: 10000.0 + prior: + input file: "{{atmosphere_obsbiasin_path}}/{{atmosphere_obsbiascovin_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovin_suffix}}" + inflation: + ratio: 1.1 + ratio for small dataset: 2.0 + output file: "{{atmosphere_obsbiasout_path}}/{{atmosphere_obsbiascovout_prefix}}{{observation_from_jcb}}{{atmosphere_obsbiascovout_suffix}}" + + # Observation Filters (QC) + # ------------------------ + obs pre filters: + # Remove Observations from the Edge of the Scan + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: MetaData/sensorScanPosition + is_in: 15-207 + action: + name: reduce obs space + + # Data Thinning + - filter: Gaussian Thinning + horizontal_mesh: 100 + use_reduced_horizontal_grid: true + distance_norm: geodesic + # round_horizontal_bin_count_to_nearest: true + # partition_longitude_bins_using_mesh: true + action: + name: reduce obs space + + # ------------------------ + obs prior filters: + # Zero Atmospheric clouds in CRTM where water_area_fraction < 0.99 + - filter: Variable Assignment + assignments: + - name: MetaData/zeroCloudInCRTM + type: int + function: + name: ObsFunction/Conditional + options: + # firstmatchingcase: false + defaultvalue: 0 # Will not zero clouds by default + cases: + - where: + - variable: + name: GeoVaLs/water_area_fraction + maxvalue: 0.99 + value: 1 # Will zero clouds by default + + obs post filters: + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: 1-9 + minvalue: 50.0 + maxvalue: 320.0 + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: 10-13 + minvalue: 70.0 + maxvalue: 320.0 + - filter: Domain Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: MetaData/latitude + minvalue: -55.0 + maxvalue: 55.0 + - variable: + name: MetaData/heightOfSurface + maxvalue: 2000 + - variable: + name: GeoVaLs/water_area_fraction + minvalue: 0.99 + - variable: + name: GeoVaLs/average_surface_temperature_within_field_of_view + minvalue: 275 + + # Toss data within a region lat: 20S - 0N deg and lon: 25E - 40E deg. (Inherited from GEOS) + - filter: BlackList + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: MetaData/latitude + minvalue: -20.0 + maxvalue: 0.0 + - variable: + name: MetaData/longitude + minvalue: 25.0 + maxvalue: 40.0 + + # Save CLW retrievals from ObsValue + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/clw_obs + type: float + function: + name: ObsFunction/CLWRetMW + options: + clwret_ch37v: 6 + clwret_ch37h: 7 + clwret_types: [ObsValue] + use bias-corrected HofX_at_ch37GHz: false + # Check CLW retrievals from ObsValue + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: DerivedMetaData/clw_obs + maxvalue: 900 + + # Save CLW retrievals from HofX + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/clw_guess_retrieval + type: float + function: + name: ObsFunction/CLWRetMW + options: + clwret_ch37v: 6 + clwret_ch37h: 7 + clwret_types: [HofX] + use bias-corrected HofX_at_ch37GHz: false + # Ckeck CLW retrievals from Hofx + - filter: Bounds Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + test variables: + - name: DerivedMetaData/clw_guess_retrieval + maxvalue: 900 + + # emiss_qc + - filter: BlackList + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: DerivedMetaData/clw_obs + minvalue: 0.0 + maxvalue: 0.05 + - variable: + name: ObsFunction/Emissivity_Diff_GMI + options: + channel: 2 + regression_constant_1: 0.13290 + regression_constant_2: 0.42468 + regression_coeff_1: [-0.00548, 0.00772, 0.00530, -0.00425, + 0.00053, 0.00008, -0.00003, -0.00144, + 0.00059, -0.00016, 0.00003, -0.00011, + 0.00017] + regression_coeff_2: [0.00289, -0.00142] + minvalue: 0.01 + action: + name: reject + - filter: BlackList + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: DerivedMetaData/clw_obs + minvalue: 0.0 + maxvalue: 0.05 + - variable: + name: ObsFunction/Emissivity_Diff_GMI + options: + channel: 4 + regression_constant_1: 0.15627 + regression_constant_2: 0.83807 + regression_coeff_1: [-0.01084, 0.01194, 0.01111, -0.00784, + 0.00060, 0.00008, -0.00003, -0.00248, + 0.00105, -0.00008, 0.00000, -0.00013, + 0.00016] + regression_coeff_2: [0.00048, -0.00207] + minvalue: 0.035 + action: + name: reject + - filter: BlackList + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: DerivedMetaData/clw_obs + minvalue: 0.0 + maxvalue: 0.05 + - variable: + name: ObsFunction/Emissivity_Diff_GMI + options: + channel: 7 + regression_constant_1: 0.30306 + regression_constant_2: 1.24071 + regression_coeff_1: [-0.01793, 0.01730, 0.01784, -0.01199, + 0.00067, 0.00013, -0.00004, -0.00365, + 0.00154, -0.00004, -0.00001, -0.00015, + 0.00017] + regression_coeff_2: [0.00068, -0.00342] + minvalue: 0.05 + action: + name: reject + + # Assign observational error in all-sky DA. + - filter: Perform Action + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + action: + name: assign error + error function: + name: ObsFunction/ObsErrorModelRamp + channels: *{{observation_from_jcb}}_simulated_channels + options: + channels: *{{observation_from_jcb}}_simulated_channels + xvar: + name: ObsFunction/CLWRetSymmetricMW + options: + clwret_ch37v: 6 + clwret_ch37h: 7 + clwret_types: [ObsValue, HofX] + use bias-corrected HofX_at_ch37GHz: false + x0: &{{observation_from_jcb}}_x0 [{{ get_satellite_variable(observation_from_jcb, "x0") }}] + x1: &{{observation_from_jcb}}_x1 [{{ get_satellite_variable(observation_from_jcb, "x1") }}] + x2: &{{observation_from_jcb}}_x2 [{{ get_satellite_variable(observation_from_jcb, "x2") }}] + err0: &{{observation_from_jcb}}_err0 [{{ get_satellite_variable(observation_from_jcb, "error") }}] + err1: &{{observation_from_jcb}}_err2 [{{ get_satellite_variable(observation_from_jcb, "error_cld1") }}] + err2: &{{observation_from_jcb}}_err1 [{{ get_satellite_variable(observation_from_jcb, "error_cld") }}] + + ## Prepare variables used by cold-air-outbreak checks in all-sky. + # Combined ColumnLiquidCloud + ColumnIceCloud + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnLiquidandIceCloud + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: GeoVaLs/mass_content_of_cloud_liquid_water_in_atmosphere_column + - name: GeoVaLs/mass_content_of_cloud_ice_in_atmosphere_column + coefs: [1, 1] + # replace zero by 9.99e-7 in DerivedMetaData/ColumnLiquidandIceCloud + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnLiquidandIceCloud + type: float + value: 9.99e-07 + where: + - variable: + name: DerivedMetaData/ColumnLiquidandIceCloud + maxvalue: 9.99e-07 + + # ratio of liquid /(Liquid+Ice) + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnRatioLiquidCloud + type: float + function: + name: ObsFunction/ElementMultiply + options: + variables: + - name: GeoVaLs/mass_content_of_cloud_liquid_water_in_atmosphere_column + - name: DerivedMetaData/ColumnLiquidandIceCloud + exponents: [1,-1] + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/ColumnRatioLiquidCloud + type: float + value: 0.0 + where: + - variable: + name: DerivedMetaData/ColumnLiquidandIceCloud + maxvalue: 1.0e-06 + + # TotalColumnVaporGuess + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/TotalColumnVaporGuess + type: float + function: + name: ObsFunction/TotalColumnVaporGuess + + # potentialTemperature at surface + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureSurface + type: float + function: + name: PotentialTemperatureFromTemperature@ObsFunction + options: + use surface pressure: true + # potentialTemperature near 700 hPa + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureAt700hPa + type: float + function: + name: PotentialTemperatureFromTemperature@ObsFunction + options: + use surface pressure: false + pressure to evaluate potential temperature: 70000.0 + # stability = Difference between PotentialTemperatur values + - filter: Variable Assignment + assignments: + - name: DerivedMetaData/PotentialTemperatureDiff + type: float + function: + name: ObsFunction/Arithmetic + options: + variables: + - name: DerivedMetaData/PotentialTemperatureAt700hPa + - name: DerivedMetaData/PotentialTemperatureSurface + coefs: [1, -1] + # cold-air-outbreak (cao) check for all-sky DA with Rain, Snow, or Graupel + - filter: BlackList + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + where: + - variable: + name: ObsFunction/TotalColumnVaporGuess + maxvalue: 8.0 + - variable: + name: DerivedMetaData/ColumnRatioLiquidCloud + maxvalue: 0.5 + - variable: + name: DerivedMetaData/PotentialTemperatureDiff + maxvalue: 12.0 + - variable: + name: GeoVaLs/water_area_fraction + minvalue: 0.99 + action: + name: reject + + # Gross check + - filter: Background Check + filter variables: + - name: brightnessTemperature + channels: *{{observation_from_jcb}}_simulated_channels + threshold: 2.0 + absolute threshold vector: [{{ get_satellite_variable(observation_from_jcb, "ermax") }}] + action: + name: reject + + # GeoVaLs for Driving Observation Operators (testing mode) + # -------------------------------------------------------- + geovals: + filename: "{{atmosphere_obsdatain_path}}/{{atmosphere_obsdatain_prefix}}{{observation_from_jcb}}_geoval{{atmosphere_obsdatain_suffix}}" + + # Passed benchmark for UFO testing + # -------------------------------- + passedBenchmark: 0 From 9e4de22fa98d524532a5807e4faaaff0c05d34a9 Mon Sep 17 00:00:00 2001 From: DavidNew-NOAA Date: Fri, 16 Jan 2026 15:28:08 -0500 Subject: [PATCH 195/199] Stop tracking submodule path/to/submodule as a submodule --- .gitmodules | 4 ---- parm/jcb-gdas | 1 - 2 files changed, 5 deletions(-) delete mode 160000 parm/jcb-gdas diff --git a/.gitmodules b/.gitmodules index 6ca2dbc18..4319b5d63 100644 --- a/.gitmodules +++ b/.gitmodules @@ -53,10 +53,6 @@ path = sorc/jcb url = https://github.com/noaa-emc/jcb.git branch = develop -[submodule "parm/jcb-gdas"] - path = parm/jcb-gdas - url = https://github.com/noaa-emc/jcb-gdas - branch = develop [submodule "parm/jcb-algorithms"] path = parm/jcb-algorithms url = https://github.com/noaa-emc/jcb-algorithms diff --git a/parm/jcb-gdas b/parm/jcb-gdas deleted file mode 160000 index 5891b215d..000000000 --- a/parm/jcb-gdas +++ /dev/null @@ -1 +0,0 @@ -Subproject commit 5891b215d82ffae1cb27aa203c6161dcc523b541 From 828ff7f62a02d0be626ccb7f2285da8fd5a98fc9 Mon Sep 17 00:00:00 2001 From: DavidNew-NOAA Date: Fri, 16 Jan 2026 20:32:40 +0000 Subject: [PATCH 196/199] Forgot to add jcb client integration test --- .github/workflows/run_jcb_basic_testing.yaml | 121 +++++++++++++++++++ 1 file changed, 121 insertions(+) create mode 100644 .github/workflows/run_jcb_basic_testing.yaml diff --git a/.github/workflows/run_jcb_basic_testing.yaml b/.github/workflows/run_jcb_basic_testing.yaml new file mode 100644 index 000000000..d8d4b1295 --- /dev/null +++ b/.github/workflows/run_jcb_basic_testing.yaml @@ -0,0 +1,121 @@ +name: Run JCB client testing with client changes + +on: + push: + branches: + - develop + pull_request: + types: + - opened + - synchronize + - reopened + +jobs: + + jcb_integration_tests: + + runs-on: ubuntu-latest + name: JCB Client Integration Tests + + env: + JCB_REPO: https://github.com/NOAA-EMC/jcb.git + + steps: + - name: Set up Python + uses: actions/setup-python@v4 + with: + python-version: '3.x' + + - name: Add repo url to the environment + run: | + JCB_APP_REPO="${{ github.repository }}" + echo "JCB_APP_REPO=${JCB_APP_REPO}" >> $GITHUB_ENV + + - name: Determine the name of the client branch + run: | + if [ "${{ github.event_name }}" == "pull_request" ]; then + JCB_APP_BRANCH=${{ github.head_ref }} + else + BRANCH_REF=${{ github.ref }} + JCB_APP_BRANCH=${BRANCH_REF#refs/heads/} + fi + echo "JCB_APP_BRANCH=$JCB_APP_BRANCH" >> $GITHUB_ENV + + - name: Determine branch to use for main jcb repo + run: | + BRANCH_NAME=${{ env.JCB_APP_BRANCH }} + if git ls-remote --heads $JCB_REPO $BRANCH_NAME | grep -q "refs/heads/$BRANCH_NAME"; then + echo "Branch $BRANCH_NAME exists in jcb repo." + echo "JCB_BRANCH=$BRANCH_NAME" >> $GITHUB_ENV + else + echo "Branch $BRANCH_NAME does not exist in jcb repo. Using develop branch." + echo "JCB_BRANCH=develop" >> $GITHUB_ENV + fi + + - name: Check for the branch name in the jcb-algorithms repo (if not develop) + run: | + BRANCH_NAME=${{ env.JCB_APP_BRANCH }} + + # If branch name is develop then we don't need to check the jcb-algorithms repo. + if [ "$BRANCH_NAME" == "develop" ]; then + echo "JCB_ALGO_BRANCH=develop" >> $GITHUB_ENV + exit 0 + fi + + # If the branch is not develop then check the jcb-algorithms repo. + if git ls-remote --heads https://github.com/NOAA-EMC/jcb-algorithms.git $BRANCH_NAME | grep -q "refs/heads/$BRANCH_NAME"; then + echo "Branch $BRANCH_NAME exists in jcb-algorithms repo." + echo "JCB_ALGO_BRANCH=$BRANCH_NAME" >> $GITHUB_ENV + + # If the branch exists in jcb-algorithms repo but JCB_BRANCH is develop then we need to + # throw an error. This is not a safe situation. The developer should create a branch in + # the main jcb repo with the same name as the branch in the jcb-application repo. + # If there were branches in other apps to account for the changes in the algorithm repo + # they could not be tested here since the scipt is not clever enough to check for + # the existence of the branch being tested here in all the clients. The safest thing to do + # is simply create a branch with the same name (even if empty) in the main jcb repo. This + # will ensure the branches of the other applications are checked out by the init script. + + if [ "${{ env.JCB_BRANCH }}" == "develop" ]; then + echo "Branch $BRANCH_NAME exists in jcb-algorithms repo but not in the main jcb repo. " + echo "Please create a branch with the same name (even if empty with no PR) in the main " + echo "jcb repo. This ensures safely checking all the clients that depend on the " + echo "jcb and jcb-algorithms repos with the changes being proposed." + exit 1 + fi + fi + + - name: Clone jcb repository + run: | + mkdir -p empty_hooks + git config --global core.hooksPath empty_hooks + git clone --branch ${{ env.JCB_BRANCH }} --recursive $JCB_REPO jcb_repo + + - name: Create custom jcb_clients.yaml for isolated testing + run: | + cd jcb_repo + + # Create a custom jcb_clients.yaml that only includes the current repository + cat > jcb_clients.yaml << EOF + gdas: + git_url: noaa-emc/gdasapp + #git_ref: develop + git_ref: feature/jcb-gdas + app_subdir: parm/jcb-gdas + EOF + + - name: Clone the clients + run: | + cd jcb_repo + pip install pyyaml + ./jcb_client_init.py + + - name: Install dependencies + run: | + cd jcb_repo + pip install .[testing] + + - name: Run the JCB client integration tests + run: | + cd jcb_repo/test/client_integration + pytest -v From 7d5a19eee849b64f80507611dfefe25b4fbca8ef Mon Sep 17 00:00:00 2001 From: DavidNew-NOAA Date: Tue, 20 Jan 2026 14:17:26 +0000 Subject: [PATCH 197/199] Clean up some things from the stand alone repo --- parm/jcb-gdas/.github/copilot-instructions.md | 171 ----- .../workflows/run_jcb_basic_testing.yaml | 120 ---- parm/jcb-gdas/.gitignore | 1 - parm/jcb-gdas/LICENSE | 674 ------------------ 4 files changed, 966 deletions(-) delete mode 100644 parm/jcb-gdas/.github/copilot-instructions.md delete mode 100644 parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml delete mode 100644 parm/jcb-gdas/.gitignore delete mode 100644 parm/jcb-gdas/LICENSE diff --git a/parm/jcb-gdas/.github/copilot-instructions.md b/parm/jcb-gdas/.github/copilot-instructions.md deleted file mode 100644 index 1d4eca1e5..000000000 --- a/parm/jcb-gdas/.github/copilot-instructions.md +++ /dev/null @@ -1,171 +0,0 @@ -# JEDI Configuration Builder - GDAS Client - -JCB-GDAS is a configuration client repository for the JEDI Configuration Builder (JCB) system used by NOAA-EMC for Global Data Assimilation System (GDAS) weather forecasting and data assimilation. It contains YAML Jinja2 templates for configuring JEDI algorithms, models, and observations. - -Always reference these instructions first and fallback to search or bash commands only when you encounter unexpected information that does not match the info here. - -## Working Effectively - -### Initial Setup and Dependencies -- Install Python 3.6+ and basic dependencies: - ```bash - python3 -m pip install --user pyyaml jinja2 click pytest - ``` - -### JCB Development Environment Setup -- Clone and set up the main JCB repository with clients: - ```bash - cd /tmp - git clone --branch develop --recursive https://github.com/NOAA-EMC/jcb.git jcb_repo # Takes 30-60 seconds - cd jcb_repo - ./jcb_client_init.py # Takes 2-3 minutes. NEVER CANCEL. Set timeout to 5+ minutes. - python3 -m pip install --user . # Takes 30-60 seconds - ``` -- The init script clones client repositories (jcb-gdas, jcb-rdas) and algorithms into `src/jcb/configuration/` -- **CRITICAL**: Always run from development directory with PYTHONPATH for local testing - -### Testing and Validation -- Run client integration tests from JCB repo with proper PYTHONPATH: - ```bash - cd /tmp/jcb_repo - PYTHONPATH=/tmp/jcb_repo/src pytest test/client_integration -v - ``` - - Takes 15-20 seconds to complete. NEVER CANCEL. - - Some tests may fail on .git directory checks - this is normal - - Set timeout to 60+ seconds for safety - -### Rendering JEDI Configurations -- Test configuration rendering using Python API: - ```bash - cd /tmp/jcb_repo - PYTHONPATH=/tmp/jcb_repo/src python3 -c " - import jcb - import yaml - config = yaml.safe_load(open('/path/to/gdas/templates.yaml')) - result = jcb.render(config) - print('Render successful') - " - ``` -- The command-line interface requires development setup: - ```bash - cd /tmp/jcb_repo - PYTHONPATH=/tmp/jcb_repo/src python3 -m jcb.driver render input.yaml output.yaml - ``` - -## Validation - -### Manual Validation Steps -- ALWAYS test configuration rendering after making changes to templates -- Run client integration tests to ensure templates are properly structured -- Validate YAML syntax for non-template files (.yaml): - ```bash - python3 -c "import yaml; yaml.safe_load(open('file.yaml'))" - ``` -- For template files (.yaml.j2), use JCB rendering to validate: - ```bash - cd /tmp/jcb_repo - PYTHONPATH=/tmp/jcb_repo/src python3 -c "import jcb, yaml; config=yaml.safe_load(open('template_dict.yaml')); jcb.render(config)" - ``` -- Check template variable consistency across related files - -### GitHub Workflow Validation -- The repository uses GitHub Actions for CI testing at `.github/workflows/run_jcb_basic_testing.yaml` -- Workflow clones main JCB repo, initializes clients, and runs integration tests -- Takes 3-5 minutes total. NEVER CANCEL. -- Set timeout to 10+ minutes for workflow commands - -## Repository Structure - -### Key Directories -- `algorithm/` - Data assimilation algorithm templates (3dvar, LETKF, etc.) - - `atmosphere/` - Atmospheric DA algorithm configurations - - `marine/`, `aero/`, `snow/` - Other domain-specific algorithms -- `model/` - Model configuration templates - - `atmosphere/` - FV3-JEDI atmospheric model configs - - Contains Jinja2 templates (.j2) for geometry, backgrounds, increments -- `observations/` - Observation operator templates - - `atmosphere/` - Satellite and conventional observation configs - - Individual files for each instrument/platform (e.g., abi_g17.yaml.j2) -- `observation_chronicle/` - Observation metadata and channel configurations -- `test/client_integration/` - Integration test templates and configurations - -### File Naming Conventions -- All template files use `.yaml.j2` extension for Jinja2 templates -- Model templates must use `_` prefix (e.g., `atmosphere_background.yaml.j2`) -- Observation files named by instrument/platform (e.g., `iasi_metop-a.yaml.j2`) - -### Template Variables -- Variables follow naming pattern: `_` -- Example: `atmosphere_background_path`, `marine_model_timestep` -- Template files in model/ directory MUST use component-prefixed variables - -## Common Tasks - -### Repository Directory Structure (for reference) -``` -/home/runner/work/jcb-gdas/jcb-gdas/ -├── .github/workflows/run_jcb_basic_testing.yaml # CI workflow -├── algorithm/ -│ ├── aero/, atmosphere/, marine/, obstats/, snow/ -│ └── atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 # Example algorithm -├── model/ -│ ├── aero/, atmosphere/, marine/, snow/ -│ └── atmosphere/atmosphere_background.yaml.j2 # Example model config -├── observations/ -│ ├── aero/, atmosphere/, atmosphere-lgetkf/, marine/, snow/ -│ └── atmosphere/iasi_metop-a.yaml.j2 # Example observation -├── observation_chronicle/ -│ ├── atmosphere/, snow/ -│ └── atmosphere/abi_g17.yaml # Example chronicle metadata -└── test/client_integration/ - ├── gdas-atmosphere-templates.yaml # Test template dictionary - └── gdas-marine-templates.yaml -``` - -### Working with Templates -- Edit `.yaml.j2` files using standard text editors -- Variables use Jinja2 syntax: `{{ variable_name }}` -- Test rendering with sample template dictionaries from `test/client_integration/` -- Always validate template syntax before committing - -### Adding New Observations -- Create new `.yaml.j2` file in appropriate `observations//` directory -- Follow existing naming patterns (instrument_platform.yaml.j2) -- Add corresponding metadata file in `observation_chronicle/` if needed -- Test rendering with existing template configurations - -### Debugging Template Issues -- Check variable naming consistency across files -- Validate YAML structure after template rendering -- Use Python API for debugging: `jcb.render(template_dict)` returns rendered config -- Review client integration test output for template validation errors - -## Important Notes - -- **NEVER CANCEL** GitHub workflow runs - they may take 5+ minutes -- **NEVER CANCEL** client initialization - takes 2-3 minutes for git clones -- **NEVER CANCEL** integration tests - they may take 15-30 seconds but can appear to hang -- This is a configuration repository - no compilation/build process required -- Changes affect JEDI experiment configurations used in weather forecasting -- Template changes require integration testing with main JCB system -- Use development JCB setup for local testing and validation - -## Troubleshooting - -### Common Issues -- **"FileNotFoundError: configuration/apps"** - Run JCB from development directory with PYTHONPATH -- **"Template key does not start with component_"** - Ensure model templates use proper prefixed variables -- **"YAML scanner error in .j2 file"** - Don't validate .j2 templates with yaml.safe_load, use JCB rendering -- **Integration tests fail on .git directory** - This is normal, test still validates templates - -### Performance Notes -- JCB client initialization: 2-3 minutes (git clones) -- Client integration tests: 15-30 seconds -- Configuration rendering: 1-5 seconds per template -- GitHub workflow: 3-5 minutes total - -### Expected File Counts -- Algorithm templates: ~10 files per component -- Model templates: ~15 files per component -- Observation templates: ~50+ files in atmosphere/ (satellites, instruments) -- Observation chronicles: ~50+ metadata files \ No newline at end of file diff --git a/parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml b/parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml deleted file mode 100644 index 8b0754e2f..000000000 --- a/parm/jcb-gdas/.github/workflows/run_jcb_basic_testing.yaml +++ /dev/null @@ -1,120 +0,0 @@ -name: Run JCB client testing with client changes - -on: - push: - branches: - - develop - pull_request: - types: - - opened - - synchronize - - reopened - -jobs: - - jcb_integration_tests: - - runs-on: ubuntu-latest - name: JCB Client Integration Tests - - env: - JCB_REPO: https://github.com/NOAA-EMC/jcb.git - - steps: - - name: Set up Python - uses: actions/setup-python@v4 - with: - python-version: '3.x' - - - name: Add repo url to the environment - run: | - JCB_APP_REPO="${{ github.repository }}" - echo "JCB_APP_REPO=${JCB_APP_REPO}" >> $GITHUB_ENV - - - name: Determine the name of the client branch - run: | - if [ "${{ github.event_name }}" == "pull_request" ]; then - JCB_APP_BRANCH=${{ github.head_ref }} - else - BRANCH_REF=${{ github.ref }} - JCB_APP_BRANCH=${BRANCH_REF#refs/heads/} - fi - echo "JCB_APP_BRANCH=$JCB_APP_BRANCH" >> $GITHUB_ENV - - - name: Determine branch to use for main jcb repo - run: | - BRANCH_NAME=${{ env.JCB_APP_BRANCH }} - if git ls-remote --heads $JCB_REPO $BRANCH_NAME | grep -q "refs/heads/$BRANCH_NAME"; then - echo "Branch $BRANCH_NAME exists in jcb repo." - echo "JCB_BRANCH=$BRANCH_NAME" >> $GITHUB_ENV - else - echo "Branch $BRANCH_NAME does not exist in jcb repo. Using develop branch." - echo "JCB_BRANCH=develop" >> $GITHUB_ENV - fi - - - name: Check for the branch name in the jcb-algorithms repo (if not develop) - run: | - BRANCH_NAME=${{ env.JCB_APP_BRANCH }} - - # If branch name is develop then we don't need to check the jcb-algorithms repo. - if [ "$BRANCH_NAME" == "develop" ]; then - echo "JCB_ALGO_BRANCH=develop" >> $GITHUB_ENV - exit 0 - fi - - # If the branch is not develop then check the jcb-algorithms repo. - if git ls-remote --heads https://github.com/NOAA-EMC/jcb-algorithms.git $BRANCH_NAME | grep -q "refs/heads/$BRANCH_NAME"; then - echo "Branch $BRANCH_NAME exists in jcb-algorithms repo." - echo "JCB_ALGO_BRANCH=$BRANCH_NAME" >> $GITHUB_ENV - - # If the branch exists in jcb-algorithms repo but JCB_BRANCH is develop then we need to - # throw an error. This is not a safe situation. The developer should create a branch in - # the main jcb repo with the same name as the branch in the jcb-application repo. - # If there were branches in other apps to account for the changes in the algorithm repo - # they could not be tested here since the scipt is not clever enough to check for - # the existence of the branch being tested here in all the clients. The safest thing to do - # is simply create a branch with the same name (even if empty) in the main jcb repo. This - # will ensure the branches of the other applications are checked out by the init script. - - if [ "${{ env.JCB_BRANCH }}" == "develop" ]; then - echo "Branch $BRANCH_NAME exists in jcb-algorithms repo but not in the main jcb repo. " - echo "Please create a branch with the same name (even if empty with no PR) in the main " - echo "jcb repo. This ensures safely checking all the clients that depend on the " - echo "jcb and jcb-algorithms repos with the changes being proposed." - exit 1 - fi - fi - - - name: Clone jcb repository - run: | - mkdir -p empty_hooks - git config --global core.hooksPath empty_hooks - git clone --branch ${{ env.JCB_BRANCH }} --recursive $JCB_REPO jcb_repo - - - name: Create custom jcb_clients.yaml for isolated testing - run: | - cd jcb_repo - # Extract the client name from the repository name (e.g., jcb-gdas -> gdas) - CLIENT_NAME=$(echo "${{ env.JCB_APP_REPO }}" | sed 's/.*jcb-//') - # Create a custom jcb_clients.yaml that only includes the current repository - cat > jcb_clients.yaml << EOF - ${CLIENT_NAME}: - git_url: ${{ env.JCB_APP_REPO }} - git_ref: ${{ env.JCB_APP_BRANCH }} - EOF - - - name: Clone the clients - run: | - cd jcb_repo - pip install pyyaml - ./jcb_client_init.py - - - name: Install dependencies - run: | - cd jcb_repo - pip install .[testing] - - - name: Run the JCB client integration tests - run: | - cd jcb_repo/test/client_integration - pytest -v diff --git a/parm/jcb-gdas/.gitignore b/parm/jcb-gdas/.gitignore deleted file mode 100644 index 5509140f2..000000000 --- a/parm/jcb-gdas/.gitignore +++ /dev/null @@ -1 +0,0 @@ -*.DS_Store diff --git a/parm/jcb-gdas/LICENSE b/parm/jcb-gdas/LICENSE deleted file mode 100644 index f288702d2..000000000 --- a/parm/jcb-gdas/LICENSE +++ /dev/null @@ -1,674 +0,0 @@ - GNU GENERAL PUBLIC LICENSE - Version 3, 29 June 2007 - - Copyright (C) 2007 Free Software Foundation, Inc. - Everyone is permitted to copy and distribute verbatim copies - of this license document, but changing it is not allowed. - - Preamble - - The GNU General Public License is a free, copyleft license for -software and other kinds of works. - - The licenses for most software and other practical works are designed -to take away your freedom to share and change the works. By contrast, -the GNU General Public License is intended to guarantee your freedom to -share and change all versions of a program--to make sure it remains free -software for all its users. We, the Free Software Foundation, use the -GNU General Public License for most of our software; it applies also to -any other work released this way by its authors. You can apply it to -your programs, too. - - When we speak of free software, we are referring to freedom, not -price. Our General Public Licenses are designed to make sure that you -have the freedom to distribute copies of free software (and charge for -them if you wish), that you receive source code or can get it if you -want it, that you can change the software or use pieces of it in new -free programs, and that you know you can do these things. - - To protect your rights, we need to prevent others from denying you -these rights or asking you to surrender the rights. Therefore, you have -certain responsibilities if you distribute copies of the software, or if -you modify it: responsibilities to respect the freedom of others. - - For example, if you distribute copies of such a program, whether -gratis or for a fee, you must pass on to the recipients the same -freedoms that you received. You must make sure that they, too, receive -or can get the source code. And you must show them these terms so they -know their rights. - - Developers that use the GNU GPL protect your rights with two steps: -(1) assert copyright on the software, and (2) offer you this License -giving you legal permission to copy, distribute and/or modify it. - - For the developers' and authors' protection, the GPL clearly explains -that there is no warranty for this free software. For both users' and -authors' sake, the GPL requires that modified versions be marked as -changed, so that their problems will not be attributed erroneously to -authors of previous versions. - - Some devices are designed to deny users access to install or run -modified versions of the software inside them, although the manufacturer -can do so. This is fundamentally incompatible with the aim of -protecting users' freedom to change the software. The systematic -pattern of such abuse occurs in the area of products for individuals to -use, which is precisely where it is most unacceptable. Therefore, we -have designed this version of the GPL to prohibit the practice for those -products. If such problems arise substantially in other domains, we -stand ready to extend this provision to those domains in future versions -of the GPL, as needed to protect the freedom of users. - - Finally, every program is threatened constantly by software patents. -States should not allow patents to restrict development and use of -software on general-purpose computers, but in those that do, we wish to -avoid the special danger that patents applied to a free program could -make it effectively proprietary. To prevent this, the GPL assures that -patents cannot be used to render the program non-free. - - The precise terms and conditions for copying, distribution and -modification follow. - - TERMS AND CONDITIONS - - 0. Definitions. - - "This License" refers to version 3 of the GNU General Public License. - - "Copyright" also means copyright-like laws that apply to other kinds of -works, such as semiconductor masks. - - "The Program" refers to any copyrightable work licensed under this -License. Each licensee is addressed as "you". "Licensees" and -"recipients" may be individuals or organizations. - - To "modify" a work means to copy from or adapt all or part of the work -in a fashion requiring copyright permission, other than the making of an -exact copy. The resulting work is called a "modified version" of the -earlier work or a work "based on" the earlier work. - - A "covered work" means either the unmodified Program or a work based -on the Program. - - To "propagate" a work means to do anything with it that, without -permission, would make you directly or secondarily liable for -infringement under applicable copyright law, except executing it on a -computer or modifying a private copy. Propagation includes copying, -distribution (with or without modification), making available to the -public, and in some countries other activities as well. - - To "convey" a work means any kind of propagation that enables other -parties to make or receive copies. Mere interaction with a user through -a computer network, with no transfer of a copy, is not conveying. - - An interactive user interface displays "Appropriate Legal Notices" -to the extent that it includes a convenient and prominently visible -feature that (1) displays an appropriate copyright notice, and (2) -tells the user that there is no warranty for the work (except to the -extent that warranties are provided), that licensees may convey the -work under this License, and how to view a copy of this License. If -the interface presents a list of user commands or options, such as a -menu, a prominent item in the list meets this criterion. - - 1. Source Code. - - The "source code" for a work means the preferred form of the work -for making modifications to it. "Object code" means any non-source -form of a work. - - A "Standard Interface" means an interface that either is an official -standard defined by a recognized standards body, or, in the case of -interfaces specified for a particular programming language, one that -is widely used among developers working in that language. - - The "System Libraries" of an executable work include anything, other -than the work as a whole, that (a) is included in the normal form of -packaging a Major Component, but which is not part of that Major -Component, and (b) serves only to enable use of the work with that -Major Component, or to implement a Standard Interface for which an -implementation is available to the public in source code form. A -"Major Component", in this context, means a major essential component -(kernel, window system, and so on) of the specific operating system -(if any) on which the executable work runs, or a compiler used to -produce the work, or an object code interpreter used to run it. - - The "Corresponding Source" for a work in object code form means all -the source code needed to generate, install, and (for an executable -work) run the object code and to modify the work, including scripts to -control those activities. However, it does not include the work's -System Libraries, or general-purpose tools or generally available free -programs which are used unmodified in performing those activities but -which are not part of the work. For example, Corresponding Source -includes interface definition files associated with source files for -the work, and the source code for shared libraries and dynamically -linked subprograms that the work is specifically designed to require, -such as by intimate data communication or control flow between those -subprograms and other parts of the work. - - The Corresponding Source need not include anything that users -can regenerate automatically from other parts of the Corresponding -Source. - - The Corresponding Source for a work in source code form is that -same work. - - 2. Basic Permissions. - - All rights granted under this License are granted for the term of -copyright on the Program, and are irrevocable provided the stated -conditions are met. This License explicitly affirms your unlimited -permission to run the unmodified Program. The output from running a -covered work is covered by this License only if the output, given its -content, constitutes a covered work. This License acknowledges your -rights of fair use or other equivalent, as provided by copyright law. - - You may make, run and propagate covered works that you do not -convey, without conditions so long as your license otherwise remains -in force. You may convey covered works to others for the sole purpose -of having them make modifications exclusively for you, or provide you -with facilities for running those works, provided that you comply with -the terms of this License in conveying all material for which you do -not control copyright. Those thus making or running the covered works -for you must do so exclusively on your behalf, under your direction -and control, on terms that prohibit them from making any copies of -your copyrighted material outside their relationship with you. - - Conveying under any other circumstances is permitted solely under -the conditions stated below. Sublicensing is not allowed; section 10 -makes it unnecessary. - - 3. Protecting Users' Legal Rights From Anti-Circumvention Law. - - No covered work shall be deemed part of an effective technological -measure under any applicable law fulfilling obligations under article -11 of the WIPO copyright treaty adopted on 20 December 1996, or -similar laws prohibiting or restricting circumvention of such -measures. - - When you convey a covered work, you waive any legal power to forbid -circumvention of technological measures to the extent such circumvention -is effected by exercising rights under this License with respect to -the covered work, and you disclaim any intention to limit operation or -modification of the work as a means of enforcing, against the work's -users, your or third parties' legal rights to forbid circumvention of -technological measures. - - 4. Conveying Verbatim Copies. - - You may convey verbatim copies of the Program's source code as you -receive it, in any medium, provided that you conspicuously and -appropriately publish on each copy an appropriate copyright notice; -keep intact all notices stating that this License and any -non-permissive terms added in accord with section 7 apply to the code; -keep intact all notices of the absence of any warranty; and give all -recipients a copy of this License along with the Program. - - You may charge any price or no price for each copy that you convey, -and you may offer support or warranty protection for a fee. - - 5. Conveying Modified Source Versions. - - You may convey a work based on the Program, or the modifications to -produce it from the Program, in the form of source code under the -terms of section 4, provided that you also meet all of these conditions: - - a) The work must carry prominent notices stating that you modified - it, and giving a relevant date. - - b) The work must carry prominent notices stating that it is - released under this License and any conditions added under section - 7. This requirement modifies the requirement in section 4 to - "keep intact all notices". - - c) You must license the entire work, as a whole, under this - License to anyone who comes into possession of a copy. This - License will therefore apply, along with any applicable section 7 - additional terms, to the whole of the work, and all its parts, - regardless of how they are packaged. This License gives no - permission to license the work in any other way, but it does not - invalidate such permission if you have separately received it. - - d) If the work has interactive user interfaces, each must display - Appropriate Legal Notices; however, if the Program has interactive - interfaces that do not display Appropriate Legal Notices, your - work need not make them do so. - - A compilation of a covered work with other separate and independent -works, which are not by their nature extensions of the covered work, -and which are not combined with it such as to form a larger program, -in or on a volume of a storage or distribution medium, is called an -"aggregate" if the compilation and its resulting copyright are not -used to limit the access or legal rights of the compilation's users -beyond what the individual works permit. Inclusion of a covered work -in an aggregate does not cause this License to apply to the other -parts of the aggregate. - - 6. Conveying Non-Source Forms. - - You may convey a covered work in object code form under the terms -of sections 4 and 5, provided that you also convey the -machine-readable Corresponding Source under the terms of this License, -in one of these ways: - - a) Convey the object code in, or embodied in, a physical product - (including a physical distribution medium), accompanied by the - Corresponding Source fixed on a durable physical medium - customarily used for software interchange. - - b) Convey the object code in, or embodied in, a physical product - (including a physical distribution medium), accompanied by a - written offer, valid for at least three years and valid for as - long as you offer spare parts or customer support for that product - model, to give anyone who possesses the object code either (1) a - copy of the Corresponding Source for all the software in the - product that is covered by this License, on a durable physical - medium customarily used for software interchange, for a price no - more than your reasonable cost of physically performing this - conveying of source, or (2) access to copy the - Corresponding Source from a network server at no charge. - - c) Convey individual copies of the object code with a copy of the - written offer to provide the Corresponding Source. This - alternative is allowed only occasionally and noncommercially, and - only if you received the object code with such an offer, in accord - with subsection 6b. - - d) Convey the object code by offering access from a designated - place (gratis or for a charge), and offer equivalent access to the - Corresponding Source in the same way through the same place at no - further charge. You need not require recipients to copy the - Corresponding Source along with the object code. If the place to - copy the object code is a network server, the Corresponding Source - may be on a different server (operated by you or a third party) - that supports equivalent copying facilities, provided you maintain - clear directions next to the object code saying where to find the - Corresponding Source. Regardless of what server hosts the - Corresponding Source, you remain obligated to ensure that it is - available for as long as needed to satisfy these requirements. - - e) Convey the object code using peer-to-peer transmission, provided - you inform other peers where the object code and Corresponding - Source of the work are being offered to the general public at no - charge under subsection 6d. - - A separable portion of the object code, whose source code is excluded -from the Corresponding Source as a System Library, need not be -included in conveying the object code work. - - A "User Product" is either (1) a "consumer product", which means any -tangible personal property which is normally used for personal, family, -or household purposes, or (2) anything designed or sold for incorporation -into a dwelling. In determining whether a product is a consumer product, -doubtful cases shall be resolved in favor of coverage. For a particular -product received by a particular user, "normally used" refers to a -typical or common use of that class of product, regardless of the status -of the particular user or of the way in which the particular user -actually uses, or expects or is expected to use, the product. A product -is a consumer product regardless of whether the product has substantial -commercial, industrial or non-consumer uses, unless such uses represent -the only significant mode of use of the product. - - "Installation Information" for a User Product means any methods, -procedures, authorization keys, or other information required to install -and execute modified versions of a covered work in that User Product from -a modified version of its Corresponding Source. The information must -suffice to ensure that the continued functioning of the modified object -code is in no case prevented or interfered with solely because -modification has been made. - - If you convey an object code work under this section in, or with, or -specifically for use in, a User Product, and the conveying occurs as -part of a transaction in which the right of possession and use of the -User Product is transferred to the recipient in perpetuity or for a -fixed term (regardless of how the transaction is characterized), the -Corresponding Source conveyed under this section must be accompanied -by the Installation Information. But this requirement does not apply -if neither you nor any third party retains the ability to install -modified object code on the User Product (for example, the work has -been installed in ROM). - - The requirement to provide Installation Information does not include a -requirement to continue to provide support service, warranty, or updates -for a work that has been modified or installed by the recipient, or for -the User Product in which it has been modified or installed. Access to a -network may be denied when the modification itself materially and -adversely affects the operation of the network or violates the rules and -protocols for communication across the network. - - Corresponding Source conveyed, and Installation Information provided, -in accord with this section must be in a format that is publicly -documented (and with an implementation available to the public in -source code form), and must require no special password or key for -unpacking, reading or copying. - - 7. Additional Terms. - - "Additional permissions" are terms that supplement the terms of this -License by making exceptions from one or more of its conditions. -Additional permissions that are applicable to the entire Program shall -be treated as though they were included in this License, to the extent -that they are valid under applicable law. If additional permissions -apply only to part of the Program, that part may be used separately -under those permissions, but the entire Program remains governed by -this License without regard to the additional permissions. - - When you convey a copy of a covered work, you may at your option -remove any additional permissions from that copy, or from any part of -it. (Additional permissions may be written to require their own -removal in certain cases when you modify the work.) You may place -additional permissions on material, added by you to a covered work, -for which you have or can give appropriate copyright permission. - - Notwithstanding any other provision of this License, for material you -add to a covered work, you may (if authorized by the copyright holders of -that material) supplement the terms of this License with terms: - - a) Disclaiming warranty or limiting liability differently from the - terms of sections 15 and 16 of this License; or - - b) Requiring preservation of specified reasonable legal notices or - author attributions in that material or in the Appropriate Legal - Notices displayed by works containing it; or - - c) Prohibiting misrepresentation of the origin of that material, or - requiring that modified versions of such material be marked in - reasonable ways as different from the original version; or - - d) Limiting the use for publicity purposes of names of licensors or - authors of the material; or - - e) Declining to grant rights under trademark law for use of some - trade names, trademarks, or service marks; or - - f) Requiring indemnification of licensors and authors of that - material by anyone who conveys the material (or modified versions of - it) with contractual assumptions of liability to the recipient, for - any liability that these contractual assumptions directly impose on - those licensors and authors. - - All other non-permissive additional terms are considered "further -restrictions" within the meaning of section 10. If the Program as you -received it, or any part of it, contains a notice stating that it is -governed by this License along with a term that is a further -restriction, you may remove that term. If a license document contains -a further restriction but permits relicensing or conveying under this -License, you may add to a covered work material governed by the terms -of that license document, provided that the further restriction does -not survive such relicensing or conveying. - - If you add terms to a covered work in accord with this section, you -must place, in the relevant source files, a statement of the -additional terms that apply to those files, or a notice indicating -where to find the applicable terms. - - Additional terms, permissive or non-permissive, may be stated in the -form of a separately written license, or stated as exceptions; -the above requirements apply either way. - - 8. Termination. - - You may not propagate or modify a covered work except as expressly -provided under this License. Any attempt otherwise to propagate or -modify it is void, and will automatically terminate your rights under -this License (including any patent licenses granted under the third -paragraph of section 11). - - However, if you cease all violation of this License, then your -license from a particular copyright holder is reinstated (a) -provisionally, unless and until the copyright holder explicitly and -finally terminates your license, and (b) permanently, if the copyright -holder fails to notify you of the violation by some reasonable means -prior to 60 days after the cessation. - - Moreover, your license from a particular copyright holder is -reinstated permanently if the copyright holder notifies you of the -violation by some reasonable means, this is the first time you have -received notice of violation of this License (for any work) from that -copyright holder, and you cure the violation prior to 30 days after -your receipt of the notice. - - Termination of your rights under this section does not terminate the -licenses of parties who have received copies or rights from you under -this License. If your rights have been terminated and not permanently -reinstated, you do not qualify to receive new licenses for the same -material under section 10. - - 9. Acceptance Not Required for Having Copies. - - You are not required to accept this License in order to receive or -run a copy of the Program. Ancillary propagation of a covered work -occurring solely as a consequence of using peer-to-peer transmission -to receive a copy likewise does not require acceptance. However, -nothing other than this License grants you permission to propagate or -modify any covered work. These actions infringe copyright if you do -not accept this License. Therefore, by modifying or propagating a -covered work, you indicate your acceptance of this License to do so. - - 10. Automatic Licensing of Downstream Recipients. - - Each time you convey a covered work, the recipient automatically -receives a license from the original licensors, to run, modify and -propagate that work, subject to this License. You are not responsible -for enforcing compliance by third parties with this License. - - An "entity transaction" is a transaction transferring control of an -organization, or substantially all assets of one, or subdividing an -organization, or merging organizations. If propagation of a covered -work results from an entity transaction, each party to that -transaction who receives a copy of the work also receives whatever -licenses to the work the party's predecessor in interest had or could -give under the previous paragraph, plus a right to possession of the -Corresponding Source of the work from the predecessor in interest, if -the predecessor has it or can get it with reasonable efforts. - - You may not impose any further restrictions on the exercise of the -rights granted or affirmed under this License. For example, you may -not impose a license fee, royalty, or other charge for exercise of -rights granted under this License, and you may not initiate litigation -(including a cross-claim or counterclaim in a lawsuit) alleging that -any patent claim is infringed by making, using, selling, offering for -sale, or importing the Program or any portion of it. - - 11. Patents. - - A "contributor" is a copyright holder who authorizes use under this -License of the Program or a work on which the Program is based. The -work thus licensed is called the contributor's "contributor version". - - A contributor's "essential patent claims" are all patent claims -owned or controlled by the contributor, whether already acquired or -hereafter acquired, that would be infringed by some manner, permitted -by this License, of making, using, or selling its contributor version, -but do not include claims that would be infringed only as a -consequence of further modification of the contributor version. For -purposes of this definition, "control" includes the right to grant -patent sublicenses in a manner consistent with the requirements of -this License. - - Each contributor grants you a non-exclusive, worldwide, royalty-free -patent license under the contributor's essential patent claims, to -make, use, sell, offer for sale, import and otherwise run, modify and -propagate the contents of its contributor version. - - In the following three paragraphs, a "patent license" is any express -agreement or commitment, however denominated, not to enforce a patent -(such as an express permission to practice a patent or covenant not to -sue for patent infringement). To "grant" such a patent license to a -party means to make such an agreement or commitment not to enforce a -patent against the party. - - If you convey a covered work, knowingly relying on a patent license, -and the Corresponding Source of the work is not available for anyone -to copy, free of charge and under the terms of this License, through a -publicly available network server or other readily accessible means, -then you must either (1) cause the Corresponding Source to be so -available, or (2) arrange to deprive yourself of the benefit of the -patent license for this particular work, or (3) arrange, in a manner -consistent with the requirements of this License, to extend the patent -license to downstream recipients. "Knowingly relying" means you have -actual knowledge that, but for the patent license, your conveying the -covered work in a country, or your recipient's use of the covered work -in a country, would infringe one or more identifiable patents in that -country that you have reason to believe are valid. - - If, pursuant to or in connection with a single transaction or -arrangement, you convey, or propagate by procuring conveyance of, a -covered work, and grant a patent license to some of the parties -receiving the covered work authorizing them to use, propagate, modify -or convey a specific copy of the covered work, then the patent license -you grant is automatically extended to all recipients of the covered -work and works based on it. - - A patent license is "discriminatory" if it does not include within -the scope of its coverage, prohibits the exercise of, or is -conditioned on the non-exercise of one or more of the rights that are -specifically granted under this License. You may not convey a covered -work if you are a party to an arrangement with a third party that is -in the business of distributing software, under which you make payment -to the third party based on the extent of your activity of conveying -the work, and under which the third party grants, to any of the -parties who would receive the covered work from you, a discriminatory -patent license (a) in connection with copies of the covered work -conveyed by you (or copies made from those copies), or (b) primarily -for and in connection with specific products or compilations that -contain the covered work, unless you entered into that arrangement, -or that patent license was granted, prior to 28 March 2007. - - Nothing in this License shall be construed as excluding or limiting -any implied license or other defenses to infringement that may -otherwise be available to you under applicable patent law. - - 12. No Surrender of Others' Freedom. - - If conditions are imposed on you (whether by court order, agreement or -otherwise) that contradict the conditions of this License, they do not -excuse you from the conditions of this License. If you cannot convey a -covered work so as to satisfy simultaneously your obligations under this -License and any other pertinent obligations, then as a consequence you may -not convey it at all. For example, if you agree to terms that obligate you -to collect a royalty for further conveying from those to whom you convey -the Program, the only way you could satisfy both those terms and this -License would be to refrain entirely from conveying the Program. - - 13. Use with the GNU Affero General Public License. - - Notwithstanding any other provision of this License, you have -permission to link or combine any covered work with a work licensed -under version 3 of the GNU Affero General Public License into a single -combined work, and to convey the resulting work. The terms of this -License will continue to apply to the part which is the covered work, -but the special requirements of the GNU Affero General Public License, -section 13, concerning interaction through a network will apply to the -combination as such. - - 14. Revised Versions of this License. - - The Free Software Foundation may publish revised and/or new versions of -the GNU General Public License from time to time. Such new versions will -be similar in spirit to the present version, but may differ in detail to -address new problems or concerns. - - Each version is given a distinguishing version number. If the -Program specifies that a certain numbered version of the GNU General -Public License "or any later version" applies to it, you have the -option of following the terms and conditions either of that numbered -version or of any later version published by the Free Software -Foundation. If the Program does not specify a version number of the -GNU General Public License, you may choose any version ever published -by the Free Software Foundation. - - If the Program specifies that a proxy can decide which future -versions of the GNU General Public License can be used, that proxy's -public statement of acceptance of a version permanently authorizes you -to choose that version for the Program. - - Later license versions may give you additional or different -permissions. However, no additional obligations are imposed on any -author or copyright holder as a result of your choosing to follow a -later version. - - 15. Disclaimer of Warranty. - - THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY -APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT -HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY -OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, -THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR -PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM -IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF -ALL NECESSARY SERVICING, REPAIR OR CORRECTION. - - 16. Limitation of Liability. - - IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING -WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS -THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY -GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE -USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF -DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD -PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), -EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF -SUCH DAMAGES. - - 17. Interpretation of Sections 15 and 16. - - If the disclaimer of warranty and limitation of liability provided -above cannot be given local legal effect according to their terms, -reviewing courts shall apply local law that most closely approximates -an absolute waiver of all civil liability in connection with the -Program, unless a warranty or assumption of liability accompanies a -copy of the Program in return for a fee. - - END OF TERMS AND CONDITIONS - - How to Apply These Terms to Your New Programs - - If you develop a new program, and you want it to be of the greatest -possible use to the public, the best way to achieve this is to make it -free software which everyone can redistribute and change under these terms. - - To do so, attach the following notices to the program. It is safest -to attach them to the start of each source file to most effectively -state the exclusion of warranty; and each file should have at least -the "copyright" line and a pointer to where the full notice is found. - - - Copyright (C) - - This program is free software: you can redistribute it and/or modify - it under the terms of the GNU General Public License as published by - the Free Software Foundation, either version 3 of the License, or - (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program. If not, see . - -Also add information on how to contact you by electronic and paper mail. - - If the program does terminal interaction, make it output a short -notice like this when it starts in an interactive mode: - - Copyright (C) - This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. - This is free software, and you are welcome to redistribute it - under certain conditions; type `show c' for details. - -The hypothetical commands `show w' and `show c' should show the appropriate -parts of the General Public License. Of course, your program's commands -might be different; for a GUI interface, you would use an "about box". - - You should also get your employer (if you work as a programmer) or school, -if any, to sign a "copyright disclaimer" for the program, if necessary. -For more information on this, and how to apply and follow the GNU GPL, see -. - - The GNU General Public License does not permit incorporating your program -into proprietary programs. If your program is a subroutine library, you -may consider it more useful to permit linking proprietary applications with -the library. If this is what you want to do, use the GNU Lesser General -Public License instead of this License. But first, please read -. From ca1c0cfc58bc57b9f302c6b17c851f21ca5425c0 Mon Sep 17 00:00:00 2001 From: DavidNew-NOAA Date: Tue, 20 Jan 2026 14:37:05 +0000 Subject: [PATCH 198/199] Add back JCB-GDAS copilot instructions --- .github/copilot-instructions.md | 172 ++++++++++++++++++++++++++++++++ 1 file changed, 172 insertions(+) create mode 100644 .github/copilot-instructions.md diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md new file mode 100644 index 000000000..2ab2406c6 --- /dev/null +++ b/.github/copilot-instructions.md @@ -0,0 +1,172 @@ +# GDASApp +## JEDI Configuration Builder - GDAS Client + +JCB-GDAS, contained in the subdirectory parm/jcb-gdas of this repository, is a configuration client for the JEDI Configuration Builder (JCB) system used by NOAA-EMC for Global Data Assimilation System (GDAS) weather forecasting and data assimilation. It contains YAML Jinja2 templates for configuring JEDI algorithms, models, and observations. + +Always reference these instructions first and fallback to search or bash commands only when you encounter unexpected information that does not match the info here. + +### Working Effectively + +#### Initial Setup and Dependencies +- Install Python 3.6+ and basic dependencies: + ```bash + python3 -m pip install --user pyyaml jinja2 click pytest + ``` + +#### JCB Development Environment Setup +- Clone and set up the main JCB repository with clients: + ```bash + cd /tmp + git clone --branch develop --recursive https://github.com/NOAA-EMC/jcb.git jcb_repo # Takes 30-60 seconds + cd jcb_repo + ./jcb_client_init.py # Takes 2-3 minutes. NEVER CANCEL. Set timeout to 5+ minutes. + python3 -m pip install --user . # Takes 30-60 seconds + ``` +- The init script clones repositories containing the clients (jcb-gdas, jcb-rdas, etc.) and algorithms into `src/jcb/configuration/` +- **CRITICAL**: Always run from development directory with PYTHONPATH for local testing + +#### Testing and Validation +- Run client integration tests from JCB repo with proper PYTHONPATH: + ```bash + cd /tmp/jcb_repo + PYTHONPATH=/tmp/jcb_repo/src pytest test/client_integration -v + ``` + - Takes 15-20 seconds to complete. NEVER CANCEL. + - Some tests may fail on .git directory checks - this is normal + - Set timeout to 60+ seconds for safety + +#### Rendering JEDI Configurations +- Test configuration rendering using Python API: + ```bash + cd /tmp/jcb_repo + PYTHONPATH=/tmp/jcb_repo/src python3 -c " + import jcb + import yaml + config = yaml.safe_load(open('/path/to/gdas/templates.yaml')) + result = jcb.render(config) + print('Render successful') + " + ``` +- The command-line interface requires development setup: + ```bash + cd /tmp/jcb_repo + PYTHONPATH=/tmp/jcb_repo/src python3 -m jcb.driver render input.yaml output.yaml + ``` + +### Validation + +#### Manual Validation Steps +- ALWAYS test configuration rendering after making changes to templates +- Run client integration tests to ensure templates are properly structured +- Validate YAML syntax for non-template files (.yaml): + ```bash + python3 -c "import yaml; yaml.safe_load(open('file.yaml'))" + ``` +- For template files (.yaml.j2), use JCB rendering to validate: + ```bash + cd /tmp/jcb_repo + PYTHONPATH=/tmp/jcb_repo/src python3 -c "import jcb, yaml; config=yaml.safe_load(open('template_dict.yaml')); jcb.render(config)" + ``` +- Check template variable consistency across related files + +#### GitHub Workflow Validation +- The JCB-GDAS client uses GitHub Actions for CI testing at `.github/workflows/run_jcb_basic_testing.yaml` +- Workflow clones main JCB repo, initializes clients, and runs integration tests +- Takes 3-5 minutes total. NEVER CANCEL. +- Set timeout to 10+ minutes for workflow commands + +### Client Structure + +#### Key Directories +- `algorithm/` - Data assimilation algorithm templates (3dvar, LETKF, etc.) + - `atmosphere/` - Atmospheric DA algorithm configurations + - `marine/`, `aero/`, `snow/` - Other domain-specific algorithms +- `model/` - Model configuration templates + - `atmosphere/` - FV3-JEDI atmospheric model configs + - Contains Jinja2 templates (.j2) for geometry, backgrounds, increments +- `observations/` - Observation operator templates + - `atmosphere/` - Satellite and conventional observation configs + - Individual files for each instrument/platform (e.g., abi_g17.yaml.j2) +- `observation_chronicle/` - Observation metadata and channel configurations +- `test/client_integration/` - Integration test templates and configurations + +#### File Naming Conventions +- All template files use `.yaml.j2` extension for Jinja2 templates +- Model templates must use `_` prefix (e.g., `atmosphere_background.yaml.j2`) +- Observation files named by instrument/platform (e.g., `iasi_metop-a.yaml.j2`) + +#### Template Variables +- Variables follow naming pattern: `_` +- Example: `atmosphere_background_path`, `marine_model_timestep` +- Template files in model/ directory MUST use component-prefixed variables + +### Common Tasks + +#### Client Directory Structure (for reference) +``` +/home/runner/work/jcb-gdas/jcb-gdas/ +├── .github/workflows/run_jcb_basic_testing.yaml # CI workflow +├── algorithm/ +│ ├── aero/, atmosphere/, marine/, obstats/, snow/ +│ └── atmosphere/fv3jedi_fv3inc_lgetkf.yaml.j2 # Example algorithm +├── model/ +│ ├── aero/, atmosphere/, marine/, snow/ +│ └── atmosphere/atmosphere_background.yaml.j2 # Example model config +├── observations/ +│ ├── aero/, atmosphere/, atmosphere-lgetkf/, marine/, snow/ +│ └── atmosphere/iasi_metop-a.yaml.j2 # Example observation +├── observation_chronicle/ +│ ├── atmosphere/, snow/ +│ └── atmosphere/abi_g17.yaml # Example chronicle metadata +└── test/client_integration/ + ├── gdas-atmosphere-templates.yaml # Test template dictionary + └── gdas-marine-templates.yaml +``` + +#### Working with Templates +- Edit `.yaml.j2` files using standard text editors +- Variables use Jinja2 syntax: `{{ variable_name }}` +- Test rendering with sample template dictionaries from `test/client_integration/` +- Always validate template syntax before committing + +#### Adding New Observations +- Create new `.yaml.j2` file in appropriate `observations//` directory +- Follow existing naming patterns (instrument_platform.yaml.j2) +- Add corresponding metadata file in `observation_chronicle/` if needed +- Test rendering with existing template configurations + +#### Debugging Template Issues +- Check variable naming consistency across files +- Validate YAML structure after template rendering +- Use Python API for debugging: `jcb.render(template_dict)` returns rendered config +- Review client integration test output for template validation errors + +### Important Notes + +- **NEVER CANCEL** GitHub workflow runs - they may take 5+ minutes +- **NEVER CANCEL** client initialization - takes 2-3 minutes for git clones +- **NEVER CANCEL** integration tests - they may take 15-30 seconds but can appear to hang +- This is a configuration client - no compilation/build process required +- Changes affect JEDI experiment configurations used in weather forecasting +- Template changes require integration testing with main JCB system +- Use development JCB setup for local testing and validation + +### Troubleshooting + +#### Common Issues +- **"FileNotFoundError: configuration/apps"** - Run JCB from development directory with PYTHONPATH +- **"Template key does not start with component_"** - Ensure model templates use proper prefixed variables +- **"YAML scanner error in .j2 file"** - Don't validate .j2 templates with yaml.safe_load, use JCB rendering +- **Integration tests fail on .git directory** - This is normal, test still validates templates + +#### Performance Notes +- JCB client initialization: 2-3 minutes (git clones) +- Client integration tests: 15-30 seconds +- Configuration rendering: 1-5 seconds per template +- GitHub workflow: 3-5 minutes total + +#### Expected File Counts +- Algorithm templates: ~10 files per component +- Model templates: ~15 files per component +- Observation templates: ~50+ files in atmosphere/ (satellites, instruments) +- Observation chronicles: ~50+ metadata files From 7fea93cc730ebdb06c63f8ac6f859dea93f496a5 Mon Sep 17 00:00:00 2001 From: DavidNew-NOAA Date: Tue, 20 Jan 2026 14:44:02 +0000 Subject: [PATCH 199/199] tweak --- .github/copilot-instructions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 2ab2406c6..82e96ff75 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -1,7 +1,7 @@ # GDASApp ## JEDI Configuration Builder - GDAS Client -JCB-GDAS, contained in the subdirectory parm/jcb-gdas of this repository, is a configuration client for the JEDI Configuration Builder (JCB) system used by NOAA-EMC for Global Data Assimilation System (GDAS) weather forecasting and data assimilation. It contains YAML Jinja2 templates for configuring JEDI algorithms, models, and observations. +JCB-GDAS, contained in the subdirectory `parm/jcb-gdas` of this repository, is a configuration client for the JEDI Configuration Builder (JCB) system used by NOAA-EMC for Global Data Assimilation System (GDAS) weather forecasting and data assimilation. It contains YAML Jinja2 templates for configuring JEDI algorithms, models, and observations. Always reference these instructions first and fallback to search or bash commands only when you encounter unexpected information that does not match the info here.