hydra-zen vs. fiddle

[terrykong]

Hello hydra-zen folks!

I'm very familiar with hydra/omegaconf, but have no familiarity with hydra-zen and the features it provides and the patterns it introduces.

I've also recently come across another project https://github.com/google/fiddle that seems to have a similar feature set that also enables building configs from python.

Is there any info on what differentiates hydra-zen from fiddle?

Thanks for your insights!

[rsokl]

Hi @terrykong thanks for bringing fiddle to my attention. I haven't seen it before.

I took a super quick look at their docs (looking at google colab notebooks on my phone in bed.. so take my assessment with whole tablespoon of salt), and here is my quick impression:

fiddle

• Is like a yaml-free version of omegaconf in the way you can create and modify configs using "builder" patterns.
• Also provides some common features with Hydra
  • it has instantiation mechanisms
  • some CLI-generation capabilities
  • json serialization of configs to record runs (I think)
• It also has some pretty heavy-duty auto-config capabilities that does fancy source code / ast inspection. Seems cool but also scary. I also think that this probably obscures your library code's type annotations if you use it.

hydra-zen
The purpose of hydra-zen is to make it easy for you to write your library code independently from Hydra, but then enable you to transparently¹ configure and run your code as Hydra jobs ². You should never have to hand-write a yaml file or create config directories when you are using hydra-zen; all configs are so-called "structured configs" (i.e. dataclasses).

hydra-zen accomplishes this by providing

• relatively simple functions for auto-generating dataclass-based configs for your liibrary's functions/classes. But more and more we are trying to ensure that users don't actually need to use these all that much.
• a custom config-store that makes it very concise to both generate and store configs for your library code.
• a custom wrapper that lets enable you to convert arbitrary functions into Hydra-compatible task functions

If you haven't seen this at a glance example, it may quickly give you a sense for how these pieces interface with your library code and with Hydra.

Summary

• Reaching for fiddle is tantamount to opting for a full replacement for both omegaconf and Hydra. You will likely need to learn some new idioms for writing code that is conducive to fiddle-based workflows, and it is unclear to me how much work/specialization that requires.
• Reaching for hydra-zen is saying "I want to use omegaconf/Hydra's features but I don't want to write any yaml files and I want to minimize the amount of Hydra-specific boilerplate that I have to deal with". Given that you are already well-familiar with Hydra/omegaconf, you will probably not need to spend much time learning about how things work.

Would be happy to provide further insights if possible

Footnotes

1. I am a total stickler for making sure that hydra-zen's patterns don't obfuscate any function signatures or type annotations. ↩

2. And thus benefit from all of Hydra/omegaconf's powerful features: CLI-generation, configuring deeply-nested parameters, swapping config groups, using job custom launchers & parameter sweepers, serialized yamls recording runs. ↩

[terrykong]

Hey @rsokl . Thanks for the thoughtful response! It gives me a lot of context and places to start learning about hydra-zen.

I'm curious if you have any thoughts on whether or not it's a fool's errand to think a fiddle->omegaconf bridge is feasible. The case in mind is if someone else has a project that uses fiddle, can their "resolved" serialized fiddle yaml configs be used, perhaps via the +experiment= pattern.

I know it isn't the aim of either project, but since both hydra and fiddle can serialize to yaml, I'm wondering if there can be interoperability via yaml configs. Perhaps this is a question for omegaconf, but I noticed that fiddle uses yaml tags, and I guess things like !fdl.Config can't be loaded unless the yaml parser is told how to parse it, but it looks soo close to something that could be converted to a compatible hydra format. Here's a sample nested fiddle Config I was looking at.

!fdl.Config
__fn_or_cls__:
  module: package.subpackage
  name: FooBar
type: int
shape: !fdl.Config
  __fn_or_cls__:
    module: numpy
    name: float32

What probably further complicates this is that there are other tags fiddle can use like fdl.Partial or fdl.TaggedValueCls.

[rsokl]

Sorry for the slow responses!

I'm curious if you have any thoughts on whether or not it's a fool's errand to think a fiddle->omegaconf bridge is feasible.

I would have to spend more time looking into fiddle to see if it is feasible. I did notice in one of their tutorials that fiddle has the ability to interoperate with some existing config systems see bottom of this page. You might see if they have looked into interop with hydra/omegaconf.