Expand the capability of handling demand flexibility in different demand sectors

[BainanXia]

🚀

• Is your feature request essential for your project?

Describe the workflow you want to enable

I wish the engine could take in demand flexibility for each demand sectors, residential, commercial, industrial, etc., separately rather than aggregating them into one demand flexibility profile on the top of a unified demand profile. In the mean time, flexibility of different demand sectors should be able to have corresponding cost associated with each of them.

Additional context

From the optimization perspective, specifying demand flexibility by sectors might not make a difference without different costs to dispatch it in different sectors. Before implementing this, we could conduct end-to-end tests (running test scenarios with demand flexibility via PowerSimData given #590 has been merged recently) by aggregating demand flexibility in all sectors into one profile. But eventually, we will need this feature to be able to conduct concrete studies and support ongoing external projects.

[danielolsen]

Without changing the structure of the inputs too much, we could probably enable this via some filename pattern-matching, e.g. instead of only looking for the files demand_flexibility_up.csv and demand_flexibility_down.csv, look for demand_flexibility_*_up.csv and demand_flexibility_*_down.csv, and create a set of demand flexibility sources based on the filenames found. Then we could have rows or columns of demand flexibility parameters in #173, and enforce constraints on each member of the flexibility set.