Store metadata for solidus resources

[shahmayur001]

Summary

This implementation extends the Solidus framework to add metadata support across multiple Spree resources. It involves database migrations, model enhancements, attribute updates, API configuration, and comprehensive testing to ensure robust functionality.

List of Resources
The following Spree resources are configured to support metadata fields:

• spree_orders
• spree_line_items
• spree_shipments
• spree_payments
• spree_stock_movements
• spree_addresses
• spree_refunds
• spree_products
• spree_customer_returns
• spree_stock_locations
• spree_store_credit_events
• spree_stores
• spree_taxonomies
• spree_taxons
• spree_variants
• spree_users
• spree_return_authorizations

Database Schema Updates

To support metadata, two new JSON/JSONB columns (public_metadata and private_metadata) are added to each of the above resources.

Migration

class AddPublicAndPrivateMetadataToSpreeResources < ActiveRecord::Migration[7.0]
  def change
    %i[
      spree_orders
      spree_line_items
      spree_shipments
      spree_payments
      spree_stock_movements
      spree_addresses
      spree_refunds
      spree_products
      spree_customer_returns
      spree_stock_locations
      spree_store_credit_events
      spree_stores
      spree_taxonomies
      spree_taxons
      spree_variants
      spree_users
      spree_return_authorizations
    ].each do |table_name|
      change_table table_name do |t|
        if t.respond_to?(:jsonb)
          add_column table_name, :public_metadata, :jsonb, default: {}
          add_column table_name, :private_metadata, :jsonb, default: {}
        else
          add_column table_name, :public_metadata, :json, default: {}
          add_column table_name, :private_metadata, :json, default: {}
        end
      end
    end
  end
end

PR Model Configuration
A shared concern Spree::Metadata is introduced to handle serialization and facilitate operations on the metadata fields.

Concern Implementation

module Spree
  module Metadata
	extend ActiveSupport::Concern

	included do
  	  store :public_metadata, coder: JSON
  	  store :private_metadata, coder: JSON
	end
  end
end

Attribute Configuration

The public_metadata and private_metadata fields are incorporated into the Spree attribute system.

Permitted Attributes

@@metadata_attributes = [private_metadata: {}, public_metadata: {}]

ATTRIBUTES = [
  :address_attributes,
  :address_book_attributes,
  :checkout_address_attributes,
  :checkout_delivery_attributes,
  :checkout_payment_attributes,
  :checkout_confirm_attributes,
  :credit_card_update_attributes,
  :customer_return_attributes,
  :image_attributes,
  :inventory_unit_attributes,
  :line_item_attributes,
  :option_type_attributes,
  :option_value_attributes,
  :payment_attributes,
  :product_attributes,
  :product_properties_attributes,
  :property_attributes,
  :return_authorization_attributes,
  :shipment_attributes,
  :source_attributes,
  :stock_item_attributes,
  :stock_location_attributes,
  :stock_movement_attributes,
  :store_attributes,
  :taxon_attributes,
  :taxonomy_attributes,
  :user_attributes,
  :variant_attributes,
  :metadata_attributes
]

Strong Parameters

def permitted_product_attributes
  permitted_attributes.product_attributes + metadata_attributes + [
	product_properties_attributes: permitted_product_properties_attributes
  ]
end

API Configuration

preference :product_attributes, :array, default: [
  :id, :name, :description, :available_on,
  :slug, :meta_description, :meta_keywords, :shipping_category_id,
  :taxon_ids, :total_on_hand, :meta_title,
  :private_metadata, :public_metadata
]

Testing

Add unit tests to verify the functionality of metadata fields.
RSpec Example

describe "metadata fields" do
  subject { described_class.new }

  it "responds to public_metadata" do
	expect(subject).to respond_to(:public_metadata)
  end

  it "responds to private_metadata" do
	expect(subject).to respond_to(:private_metadata)
  end

  it "can store data in public_metadata" do
	subject.public_metadata = { "delivery_required" => "no" }
	expect(subject.public_metadata["delivery_required"]).to eq("no")
  end

  it "can store data in private_metadata" do
	subject.private_metadata = { "preferred_delivery_time" => "n/a" }
	expect(subject.private_metadata["preferred_delivery_time"]).to eq("n/a")
  end
end

Checklist

Check out our PR guidelines for more details.

The following are mandatory for all PRs:

• I have written a thorough PR description.
• I have kept my commits small and atomic.
• I have used clear, explanatory commit messages.

The following are not always needed:

• 📖 I have updated the README to account for my changes.
• 📑 I have documented new code with YARD.
• 🛣️ I have opened a PR to update the guides.
• ✅ I have added automated tests to cover my changes.
• 📸 I have attached screenshots to demo visual changes.

[fthobe]

Aims at solving #5951

[boomer196]

Aims at solving #5951

Thanks for submitting this to the team! It is something I was hoping for as well. I have been watching the issue #5897 you opened.

[fthobe]

Aims at solving #5951

Thanks for submitting this to the team! It is something I was hoping for as well. I have been watching the issue #5897 you opened.

Hey,
could you elaborate on your use cases in a couple of lines?
There are some discussions on the various ends and means of this PR.

[boomer196]

Aims at solving #5951

Thanks for submitting this to the team! It is something I was hoping for as well. I have been watching the issue #5897 you opened.

Hey, could you elaborate on your use cases in a couple of lines? There are some discussions on the various ends and means of this PR.

Sure, basically I am hoping to have Solidus support something similar to the Metafields concept that Shopify has. I am just getting started with the platform, so I still have lots to learn, but when I saw the issue you opened, I felt like the "metadata" concept was something that would be a great addition to the core. I personally like the idea of having somewhere to store "structured" data on these objects that the core system doesn't really need to care about, but that storefronts can utilize.

[fthobe]

Sure, basically I am hoping to have Solidus support something similar to the Metafields concept that Shopify has. I am just getting started with the platform, so I still have lots to learn, but when I saw the issue you opened, I felt like the "metadata" concept was something that would be a great addition to the core. I personally like the idea of having somewhere to store "structured" data on these objects that the core system doesn't really need to care about, but that storefronts can utilize.

What resources do you think need it most and why?

[jarednorman]

Product, variant, order, line item, and shipment would be reasonable defaults. They are the most important core models and my experience they are the records that need additional fields in the broadest number of cases.

[fthobe]

Product, variant, order, line item, and shipment would be reasonable defaults. They are the most important core models and my experience they are the records that need additional fields in the broadest number of cases.

Ok, given your pref for products and transactions also returns and RMA requests make sense at this point. Can you live with them + users and I stop bugging you?

[kennyadsl]

Honestly, I don't think it's super important on which models we will add those metadata fields for the core. What's more important is creating a developer-friendly journey to add more of them when needed. What I envision for the success of this initiative is:

• An easy way to add this metadata on any model. What about something like rails generate solidus:add_metadata Spree::Product? This task will generate a migration that adds the jsonb column to the corresponding table. It will probably do more with time, but that's already a good start.
• Create a solid definition for the metadata. It might contain the type and the access level, something like:

{
  "key": "serial_number",
  "type": "string",
  "value": "123abc",
  "access": "public", // Can also be the roles which have permissions, like "admin"?
  "api": true // If we want this field to be included in the REST API payloads
}

• Admin UI for resources with the metadata field. When the metadata is present, the Admin UI is smart enough to add a new section that allows operators to set those fields, with a different rendering based on their type. See what we did in the legacy admin for preferences here. Something similar would work, and can be extended if there are custom needs of extra types.

With these three elements, we have enough flexibility and we don't have to assume anything on where the fields are needed, which is without doubt something that dramatically changes from store to store.

[fthobe]

@kennyadsl i really feel strongly about the admin part being a @MadelineCollier task. She's deep into that right now and it would take us much more time to do that. Can we slip that into her existing work?

[kennyadsl]

She's deep into that right now and it would take us much more time to do that. Can we slip that into her existing work?

Yes, but it needs to be prioritized with the rest of the tasks in that case. By the way, the feature can live without the admin part, and we can add that later, isn't it?

[fthobe]

For me it's an API feature with read only admin

[fthobe]

Product, variant, order, line item, and shipment would be reasonable defaults. They are the most important core models and my experience they are the records that need additional fields in the broadest number of cases.

Given how addresses are made implications should be documented. For how it works now, would two addresses with different metadata create two different addresses or one address? @jarednorman

[jarednorman]

Addresses with the same fields will always be the same record, so you can't really attach metadata to them that's specific to a particular use of that address.

[fthobe]

Yes, but it needs to be prioritized with the rest of the tasks in that case. By the way, the feature can live without the admin part, and we can add that later, isn't it?

How do you usually handle regional differences in address paterns (First Name Last Name for example is required for legal reasons in Italy if you want to create an invoice.

[fthobe]

@shahmayur001 please remove the following ressources.

• spree_addresses
• spree_taxonomies
• spree_taxons

[jarednorman]

Addresses have a unified name field and that's part of what makes up an address, so it's very unlikely that an address ever actually gets reused by different users (unless they are different accounts/orders created by the same person) as @kennyadsl alluded to in our call.

[kennyadsl]

If you need to split first and last name, you can use something like https://github.com/basecamp/name_of_person where needed. In my experience, this is enough for 99% of the occurrences, especially in Italy where there are few name exceptions.

[tvdeyen]

If you need to split first and last name, you can use something like https://github.com/basecamp/name_of_person where needed. In my experience, this is enough for 99% of the occurrences, especially in Italy where there are few name exceptions.

There is also the Spree::Address::Name class we ship in core, that splits first_name and last_name by first space. https://github.com/solidusio/solidus/blob/main/core/app/models/spree/address/name.rb

[fthobe]

Honestly, I don't think it's super important on which models we will add those metadata fields for the core. What's more important is creating a developer-friendly journey to add more of them when needed. What I envision for the success of this initiative is:

• An easy way to add this metadata on any model. What about something like rails generate solidus:add_metadata Spree::Product? This task will generate a migration that adds the jsonb column to the corresponding table. It will probably do more with time, but that's already a good start.

• Create a solid definition for the metadata. It might contain the type and the access level, something like:

{

  "key": "serial_number",

  "type": "string",

  "value": "123abc",

  "access": "public", // Can also be the roles which have permissions, like "admin"?

  "api": true // If we want this field to be included in the REST API payloads

}

• Admin UI for resources with the metadata field. When the metadata is present, the Admin UI is smart enough to add a new section that allows operators to set those fields, with a different rendering based on their type. See what we did in the legacy admin for preferences here. Something similar would work, and can be extended if there are custom needs of extra types.

With these three elements, we have enough flexibility and we don't have to assume anything on where the fields are needed, which is without doubt something that dramatically changes from store to store.

Hey, in that case i would stash it in a table and add an Interface, but to me it seems we find more and more overhead to integrate for something that was going to leave minimal integration work and shift again something over to implementation instead of having something ready to use. I am not sure that that is the right way to go. @kennyadsl can you make a more lightweight ready to use suggestion?

[fthobe]

if someone has a proposal that does not require a full rewrite and allows customer facing as well admin facing metadata, we are totally up for that.

[fthobe]

Guys, we really like solidus, we have contributed brands, several front-end fixes, just finished up subscriptions, have offered to contribute financially to fixing up license issues and want to keep going on reviews and sitemap to fix those up too with an eye on GA4 and structured data, but there must be a clear direction and not burning our hours as well as yours.

I strongly believe rendering solidus more complete makes sense of it's own (GA4, Structured Data, Brands, Reviews,....) and we are perfectly happy to contribute to that, but there must be some flexibility on your side.

For this feature to make sense for us we need to have the possibility to insert fully dynamic key value pairs, I don't feel that manual configuration (as suggested by @kennyadsl) is adding anything here. As @jarednorman said, if you three deem it not acceptable it's not gonna be part of the core. We are absolutely willing to kick in a resource to sponsor the development and continue fixing up extensions we need, but it must make sense for everybody. For us a system where we have to configure values instead of writing random values does not make sense, it's really a deal breaker for us (even if you deem that a useless feature). We are absolutely willing to comply with any strategy you three come up with to allow:

• customers to write any key pair value (with limitations regarding quantity and content of key value pairs) into orders, line items and subscriptions;
• admins (via api) write any key pair value in any transactional type (everything that moves money or products in any way) in addition to products and all kind of line items (subscriptions, products).

It will trickle down with support for other extensions we want to fix (such as Multi Store and subscriptions), but we have the need for a clear road.

I ask you three to figure out what you want to do here, a hard no is absolutely fine for me. I think Solidus is a great product, but to receive more contributions back from existing customers it needs to guide more than it does now (on ecommerce basics like SEO) and go less defensive regarding what's expected from the customer. Right now no matter if I open the demo store or any extension it's broken left and right, we would like to help with that, but meta data is a core feature for us in the way we envisioned it (Public and Private with dynamic key value pairs). If you feel that's too much of a stretch that's fine, but I don't feel we have been lacking will to contribute.

I am looking forward to your reply, whichever it may be.

[kennyadsl]

Hey @fthobe, I understand the frustration but as I mentioned already, this kind of structural changes might take time. We don't want to merge a solution in core just because it's ready, or another project implemented the feature in this way, or because one company needs the feature ASAP.

For example, @mamhoff (sorry for the ping here), one of the most skilled and experienced engineer we have in our lucky community, took ~1 year to refine the new promotion system, and more or less the same amount of time in the past to improve the pricing system. He did that by submitting PRs, getting feedback and reiterating patiently. For both, we have stable, robust, and flexible implementations now, which don't require a lot of maintenance.

I'm sorry if it might sound harsh, but we need to put the maintainability of the platform in front of everything, because we are a small team as you can see, and we have a job. No one in the core team is paid to work on Solidus. That's also why there might be a delay in answering discussions, and commenting over and over again has the opposite effect: giving us more things to read in the amount of time we can dedicate to Solidus.

That said: again, I'm not saying we need to implement the solution as I proposed, but I'm waiting for some constructive comment around why what you copy/pasted from Spree is better abd still have to see some real reasoning that is not: "we need it this way" or "we don't want to re-do the work again".

I'm sorry but this is not the way we merged things in core so far and (I think I can talk on behalf of the @solidusio/core-team here), we want to stick with this approach. As the core team, we are responsible for finding the thin line between what can be merged and what needs more work.

Please, be patient. And if you need this immediately, just do it in your project, there's nothing bad with that. If this PR goes in core as is, only have to remove the custom code and it will keep working. If it will look like something else you can both keep your solution or migrate to the new one.

Final thoughts: thanks for contributing, we appreciate the effort you are putting into upstreaming the work, which is not common nowadays.

[tvdeyen]

Very well said, Alberto.

I think your proposal seems to be a good solution to the problem. I like the idea of using user roles as access control of attributes to return.

The proposal in this PR is not I would support ending up in core. There is no need in having two sets of attributes, if we can use Solidus' access control (provided by CanCanCan) for returning just the attributes for the particular user requesting. Something that we already do for other attributes.

[fthobe]

(Edited out some typos)
@tvdeyen we have at all length requested on slack as well as on GitHub clear guidance on how we can make this work. I invite both of you to provide clear directions on how dynamic key value pairs can be accepted for users and administrative roles and we have received no feedback on that at all. What we got is a new iteration of custom fields pushing the management burden from fields to roles.

My problem is that I have no idea what you three are governance wise talking about: most extensions fail circle ci, most do not work, license issues are plenty which rendered any installation (also from a legal standpoint) of extensions hard to impossible on current code base. The only feedback I receive are either changing feature specifications or rebuffs.

I find it personally very hard to believe that our will to fix a magnitude of extensions is outweighted by our only hard requirement to have public and private meta data in a setting that allows dynamic values, which are literally the only request we have to sustain feature wise a contribution. Right now, to be frank, I have the impression that we can contribute only features that align with the maintainer interests and not community interests which renders for me any discussion that I had with @tvdeyen and @kennyadsl moot at best, especially regarding competition to Shopify and a lack of return PRs from the community, what I saw till today is, with some very notable exceptions from @kennyadsl, exactly handbook behaviour on how to not do it. I am absolutely sure that all of you three have technical competencies that far outweighs ours on this topic right now, but I also believe that you three receive exactly the level of contribution right now that is appropriate for the approach of community management you currently drive. Which is fine, just write it on the website and the readme, because right now I have significant struggles to see any coordinated effort or decision processes that are open or public (I at least couldn’t find one on the new promotion system which to my understanding was a sponsored effort by candlescience).

I renew this one last time: find a way to work with us on the meta data topic or continue a politic of closing down this application, the result in my opinion will just be very similar to what’s present right now, a community majorly driven by three companies used as a sandbox product. I find that very sad and in stark conflict to what I have heard from you about requiring a better community. We are not asking you to merge anything you feel uncomfortable with, we are asking you to help us align what we need with what you want.

[fthobe]

This is by the way an email I received:

On Fri, 6 Dec 2024 at 20:29, wrote:
Hi Fabian,

I submitted those PRs on Solidus Digital Downloads. I kept them very specific and small and only addressed the “main” issues to get it running in v4.4. I am going to see if anything comes out of them, as I am not only evaluating the software, but the repo owners.

Anyway, I just saw you comment on the metadata PR that you submitted and I thought it was spot on, so thank you for making it. I have worked on/with many open source projects and sometimes the maintainers are happy to get community involvement and sometimes not. It sounds like you guys have been using Solidus enough that you are pretty vested…. Do you mind sharing some insight on your experience and really your candid thoughts?

I can say that with my limited experience, I have noticed some things that give me pause:

• It seems as though many things are broken and not necessarily maintained. Which, would be fine if they let the community help. Have you had a similar experience?
• it appears as they have no issues getting code merged that “they" write and some of it without even testing it. That might be a little unfair, but I have found several bugs that was basic functionality, that IMO, the dev should have caught unless they never actually tested it in the application (more than a basic unit test).

I have also found some things that are a huge positive for me:

• if you mention an issue on slack it is literally coded and merged within a couple of hours, so assuming it is a good fix, that is a great thing.
• You get a lot of stuff out of the box, which is great. I am looking for something that can just work and not be a ton of maintenance for me. (medium size store with a single dev)

Also, if you don’t mind I’d like to ask a couple of specific questions:
Have you looked at Spree and if so what were your thoughts? I know Solidus was a fork of spree.
What are you thoughts on the actual code in the “core”. Specifically, do you think it is just a learning curve or do you find it complex or hard to maintain? (that is a bit of a loaded question LOL )

Sorry for the long winded email, but I would love to get some of your thoughts as you have spent much more time with the platform.

Thanks,
Removed

[kennyadsl]

@fthobe I don't think the core team doesn't want to have a discussion on this feature. Quite the opposite is happening, actually.

No one in the core team needs this feature and still, we are here discussing and trying to help shape a better form of it, which is in contrast with your "I have the impression that we can contribute only features that align with the maintainer interests and not community interests", isn't it?.

The only kind of discussion happening is: merge it as is or we'll stop contributing, without even trying to argument why the proposed version is a good/better implementation. I don't understand how we went into this "blackmail" mode, honestly but this is not how I want to interact with the community. I am part of the core team member because I enjoy working on open-source projects and interacting with people, and this is not enjoyable.

I'll get back to this issue/PR when the mood of the conversation changes.

Regarding the status of the ecosystem, we have a lot that can be improved, true. I don't think we will improve that by merging anything that is proposed, blindly.

[fthobe]

@kennyadsl again, all of you three can feel free to make a suggestion

1. that maintains the character of the feature
2. contains a road to completition

We will as we have in the past positively evaluate it. If all we get is continuously changing specs with continuously changing instructions we will not byte.

Transport spree meta was always seen by all sides as flawed, there was no discussion there. We are absolutely willing to work with you on that if you give clear instructions that maintain the desired outcome, right now I find the process in which you accept community contributions very flawed. If you or any of the other two want to give that a try and come up clear instructions or an alternative that creates the same outcome we will be all yours. If the process continues to be the same of gradually burn resources and change road mid project we won't.

[kennyadsl]

I told you about my doubts on copy/pasting what Spree did since the first discussion we had, isn't it? Still, you moved on with doing that without trying to design any other possible solution. You really can't blame us for burning resources.

Please @fthobe, be clear: is copy/pasting this feature from Spree the only possible outcome we can have here? That's understandable but if that's the case, I think it's fair to stop discussing any other unrelated governance issue and go straight to the point: there's a clear lack of will to think and design an alternative solution unless this work comes from someone else.

If that's not the case, I'm totally open to continuing the discussion and please tell me why we should go with this design, which again, I'm not totally against. I just would like to constructively understand if we have a better path, together with who has this need and probably has the best product vision of the feature.

[fthobe]

Edit 1: Public Catalogue Metadata is rendered optional
Edit 2: An example of customer meta data has been integrated

Disclaimer

@kennyadsl This is the best I can do.
I do believe there's a middle ground to be reached here! No, copy and pasting is not the only solution, I find the spree solution suboptimal but sufficient. I do admit it lacks in many aspects and mixes three things that should not be mixed:

1. 3rd Party Integration information;
2. Customisations of the cart by customers;
3. Enriching the catalogue to improve the user journey or suffice to legal or operational requirements.

What I think it does very well is giving a universal backend for information and that's the outcome I try to focus on. I do believe TBH that three kind of information are needed (see below):

1. Administrative meta data reserved to Admin Roles
2. Customer meta data to be created during purchase by customers accessible and writable by admin roles
3. Public Catalogue meta data to enrich the catalogue with data required from a legal, operational or marketing point of view with the same set of read only public access as the rest of the customer facing catalogue information.

I have broken down the entire scope below. If you people can decide upon how to do it I would even go as far as to trash the existing solution and start from scratch but the outcome needs to be clear. I strongly believe you people do know best how this can be achieved and we are absolutely willing to sponsor the whole development back to back including documentation, but I also strongly believe that the outcome should allow to stash pretty much anything that fits into a json array and leave application to the operator of solidus, but most of all be clear from the beginning. I added a comment to the end in merits of all the policy discussions. What I wrote down here contains:

• the maximum dream (the full tables);
• what I personally deem reasonable and a middle ground (bold rows).

If the specs below are an outcome that you feel you can support we are all for it to getting back to the drawing board if we get a reasonable amount of information during the process to allow non redundant work for us and a reasonable solution for you.

Types of Meta Data and why we need it

Overview

I have broken meta data down in three large topics visible in the table below.

	Use Case
Administrative Metadata	Information required to correlate all transactions to 3rd party systems.
Customer Metadata	Information allowing customers to customize orders (eg options to store information like a text string or link to an image) or correlate transactions (such as POs).
Public Metadata	Add data to make the catalogue comply with either legal requirements (for example chemical composition, security warnings,...), operational requirements (such as GTIN,...) .

Administrative Metadata

We do not believe that solidus is used anywhere as standalone system (edge cases excluded of course). Transactional resources are either by nature integrated with 3rd party systems (eg Payments, Billing in Italy with eInvoicing) or by operational requirement (ERP, CRMs like Salesforce,...). For that purpose I see meta data as a senseful extension on admin site for all transactional ressources (also payments, stock movements (even if we won't use them) and RMAs. Given that more and more countries switch to eInvoicing (Germany is next and US has had testing phases for B2B in 2022) in the coming years I personally (and I might well be alone with that opinion) deem meta data on orders a hard requirement to future proof the system.

Customer Meta Data

Customer meta data allows users to customize a cart (incomplete order) with either administrative (PO numbers,...) or customisation information (eg an URL to a picture to customize a shirt, text strings for incisions or in our case IMEIs to relate to a subscription) allowing to significantly extend the features without changing the underlying model. Fields should be accessible on front-end and backend.

(Public) Catalogue Meta Data

Data that allows to extend the product catalogue to comply with legal requirements or marketing requirements well aware that surely different use cases will be hacked into it by others in the future. Given that the name Public Meta Data has been sufficiently burned I called it Catalogue Meta Data.

Schema

Meta Data should net a reliable outcome:
"key1" : "value1" should be as applicable as "key2" : "value2", without restriction other than the quantity of storable key value pairs and limitations of the dimension of the value given that json as a format does not pose limits we should do that server side. I think it's a front-end job to render the information correctly for the public.

Implementation

Make a decent suggestion and we will deliver it, I think it's important to provide a logic to enable and disable this feature. There can be different approaches:

• we stash a jsonb as suggested by jared in an additional column
• A system wide polymorphic solution with an independent table that belongs to many records, with a boolean column for public/private and a helper method in there to validate on call.

Scopes & Arrays

Nature of the Values

The system should be agnostic to the value contained. A decision has been made to store the data in additional columns in a jsonb which by now is relatively widely supported. Some preexisting installations might have to upgrade DBs (I am looking at SQLite).

"key1" : "value1" should lead to the same behaviour as "key2" : "value2" without requiring modification inside the code. Reasonable provisions should be taken inside the code base to edit the quantities and content size of key value pairs and appropriate errors should be returned where rate or size limits are exceeded.

Administrative Meta Data

Allow appropriate roles to customize resources with private fields not end-user accessible.
R/W = Read and Write Access

	Admin	Use Case
Order	R/W	Add 3rd party system reference fields such as PO-Order or Dropshipping reference
Line Item	R/W	Add information to customize object such as link to an image for customization of a product or relate to a 3rd party system reference
RMA	R/W	Add 3rd party system reference fields such as PO-Order or Dropshipping reference
Payment	R/W	Add 3rd party system reference fields such as PO-Order or Dropshipping reference
Order	R/W	Add 3rd party system reference fields such as PO-Order or Dropshipping reference
Shipment	R/W	Add 3rd party system reference fields such as a delivery note
Payment	R/W	Add 3rd party system reference fields for payment conciliation

Customer Meta Data

Allowing users to customize an incomplete order (or I'd say a cart) and call the information after a completed order. Admins should in an ideal case be allowed to edit those information after purchase, but I feel that can wait if technical limitations are currently in place.

	User	Admin	Use Case
Order	R/W	R/W	Add 3rd party system reference fields such as PO-Order,
Line Item	R/W	R/W	Add information to customize object such as link to an image for customization of a product or relate to a 3rd party system reference
User	R/W	R/W	Add additional User Fields or References

Public Catalogue Meta Data

Fields that can be manipulated by administrative roles and are read only. Solidus is extremely weak chested on SEO and deserves more ways to stash data.

	Admin	Use Case
Products	R/W	Add 3rd party system reference fields such EAN or GTIN or other structured data fields, add warranty or RMA terms, add links to legally required documents (eg environmental specifications)
Variants	R/W	Add 3rd party system reference fields such EAN or GTIN or other structured data fields, add warranty or RMA terms, add links to legally required documents (eg environmental specifications)
Store	R/W	Add structured data information required to comply with https://schema.org/OnlineStore

Personal Comment

I personally do not believe that Shopify discarded meta fields for any other reason than the platform being used as a "store it all" backend for applications that were not financially viable for shopify given their pricing structure. They were by the way substituted by application meta fields which are alive and well in the current api for usage on front-end and backend, but that barriers have been in my again personal opinion be included to make this a more premium tier oriented feature for sales channels such as their partners.

I think all of you three are very smart cookies (and I really believe that) and from what I heard mamhoff is smarter than all of us combined, but I really believe you people need a strong rain check. You need to make up your mind if this repo is a sandbox for your own products or an open platform (which right now it's not) and advertise it accordingly. Right now I do not personally see how the current approach is going to net you what you want (from a business perspective as well as community perspective) and I personally prefer drawing lines and provide clarity straight from the beginning and break bread over this topic or don't break bread at all. There's absolutely no obligation for you to follow us on this, but there's also absolutely no obligation for us to give back where it doesn't make sense for all parties. Again, all under the premise that if the outcome is reasonable we are happy to contribute and bear the costs for the contribution, clearer than that I can't phrase it. We have without hesitation contributed to several extensions and legal topics already and are happy to continue if you open up the decision making process beyond a simple I do not like it, otherwise it's simply spree maintained by three parties instead of one for us.

Example 1: Create order

{
  "order": {
    "email": "[email protected]",
    "customer_order_meta": [{
      "phone":"1234567890123456",
      "contract":"ref_123456"
      }],
    "line_items_attributes": [
      {
        "quantity": 1,
        "variant_id": 33,
        "customer_line_item_meta": [{
          "imei":"1234567890123456"
          }],   
      }]
    }
  }

Example Reponse 1: Create order

{
  "additional_tax_total": "string",
  "adjustment_total": "string",
  "adjustments": [
    {
      "adjustable_id": 0,
      "adjustable_type": "string",
      "amount": "string",
      "created_at": "string",
      "display_amount": "string",
      "finalized": true,
      "id": 0,
      "label": "string",
      "source_id": 0,
      "source_type": "string",
      "updated_at": "string"
    }
  ],
  "bill_address": {
    "address1": "string",
    "address2": "string",
    "alternative_phone": "string",
    "city": "string",
    "company": "string",
    "country": {
      "id": 0,
      "iso": "string",
      "iso3": "string",
      "iso_name": "string",
      "name": "string",
      "numcode": 0
    },
    "country_id": 0,
    "country_iso": "string",
    "id": 0,
    "name": "string",
    "phone": "string",
    "state": {
      "abbr": "string",
      "country_id": 0,
      "id": 0,
      "name": "string"
    },
    "state_id": 0,
    "state_name": "string",
    "state_text": "string",
    "zipcode": "string"
  },
  "canceler_id": 0,
  "channel": "string",
  "checkout_steps": [
    "string"
  ],
  "completed_at": "string",
  "covered_by_store_credit": true,
  "created_at": "string",
  "credit_cards": [
    {
      "address": null,
      "cc_type": "string",
      "id": 0,
      "last_digits": "string",
      "month": "string",
      "name": "string",
      "year": "string"
    }
  ],
    {
      "adjustments": [
        {
          "adjustable_id": 0,
          "adjustable_type": "string",
          "amount": "string",
          "created_at": "string",
          "display_amount": "string",
          "finalized": true,
          "id": 0,
          "label": "string",
          "source_id": 0,
          "source_type": "string",
          "updated_at": "string"
        }
      ],
      "display_amount": "string",
      "id": 0,
      "price": "string",
      "quantity": 0,
      "single_display_amount": "string",
      "total": "string",
      "variant": {
        "cost_price": "string",
        "depth": "string",
        "description": "string",
        "display_price": "string",
        "height": "string",
        "id": 0,
        "images": [
          {
            "alt": "string",
            "attachment_content_type": "string",
            "attachment_file_name": "string",
            "attachment_height": 0,
            "attachment_updated_at": "string",
            "attachment_width": 0,
            "id": 0,
            "large_url": "string",
            "mini_url": "string",
            "position": 0,
            "product_url": "string",
            "small_url": "string",
            "type": "string",
            "viewable_id": 0,
            "viewable_type": "string"
          }
        ],
        "in_stock": true,
        "is_backorderable": true,
        "is_destroyed": true,
        "is_master": true,
        "name": "string",
        "option_values": [
          {
            "id": 0,
            "name": "string",
            "option_type_id": 0,
            "option_type_name": "string",
            "option_type_presentation": "string",
            "presentation": "string"
          }
        ],
        "options_text": "string",
        "price": "string",
        "sku": "string",
        "slug": "string",
        "total_on_hand": 0,
        "track_inventory": true,
        "weight": "string",
        "width": "string"
      },
      "variant_id": 0,
      "customer_line_item_meta": [{
        "imei":"1234567890123456"
        }],
    }
  ],
  "number": "string",
  "order_total_after_store_credit": "string",
  "payment_methods": [
    {
      "id": 0,
      "method_type": "string",
      "name": "string",
      "partial_name": "string"
    }
  ],
  "payment_state": "string",
  "payment_total": "string",
  "payments": [
    {
      "amount": "string",
      "avs_response": "string",
      "created_at": "string",
      "display_amount": "string",
      "id": 0,
      "payment_method": {
        "id": 0,
        "method_type": "string",
        "name": "string",
        "partial_name": "string"
      },
      "payment_method_id": 0,
      "source": {
        "cc_type": "string",
        "gateway_customer_profile_id": "string",
        "gateway_payment_profile_id": "string",
        "id": 0,
        "last_digits": "string",
        "month": "string",
        "name": "string",
        "year": "string"
      },
      "source_id": 0,
      "source_type": 0,
      "state": "string",
      "updated_at": "string"
    }
  ],
  "permissions": {
    "can_update": true
  },
  "ship_address": {
    "address1": "string",
    "address2": "string",
    "alternative_phone": "string",
    "city": "string",
    "company": "string",
    "country": {
      "id": 0,
      "iso": "string",
      "iso3": "string",
      "iso_name": "string",
      "name": "string",
      "numcode": 0
    },
    "country_id": 0,
    "country_iso": "string",
    "id": 0,
    "name": "string",
    "phone": "string",
    "state": {
      "abbr": "string",
      "country_id": 0,
      "id": 0,
      "name": "string"
    },
    "state_id": 0,
    "state_name": "string",
    "state_text": "string",
    "zipcode": "string"
  },
  "ship_total": "string",
  "shipment_state": "string",
  "shipments": [
    {
      "adjustments": [
        {
          "adjustable_id": 0,
          "adjustable_type": "string",
          "amount": "string",
          "created_at": "string",
          "display_amount": "string",
          "finalized": true,
          "id": 0,
          "label": "string",
          "source_id": 0,
          "source_type": "string",
          "updated_at": "string"
        }
      ],
      "cost": "string",
      "id": 0,
      "manifest": [
        [
          {
            "quantity": 0,
            "states": {
              "on_hand": 0
            },
            "variant_id": 0
          }
        ]
      ],
      "number": "string",
      "order_id": "string",
      "customer_order_meta": [{
          "phone":"1234567890123456",
          "contract":"ref_123456"
          }],
      "selected_shipping_rate": {
        "cost": "string",
        "display_cost": "string",
        "id": 0,
        "name": "string",
        "selected": true,
        "shipping_method_code": "string",
        "shipping_method_id": 0
      },
      "shipped_at": "string",
      "shipping_methods": [
        {
          "code": "string",
          "id": 0,
          "name": "string",
          "shipping_categories": [
            {
              "id": 0,
              "name": "string"
            }
          ],
          "zones": [
            {
              "description": "string",
              "id": 0,
              "name": "string"
            }
          ]
        }
      ],
      "shipping_rates": [
        {
          "cost": "string",
          "display_cost": "string",
          "id": 0,
          "name": "string",
          "selected": true,
          "shipping_method_code": "string",
          "shipping_method_id": 0
        }
      ],
      "state": "string",
      "stock_location_name": "string",
      "tracking": "string",
      "tracking_url": "string"
    }
  ],
  "special_instructions": "string",
  "state": "string",
  "tax_total": "string",
  "token": "string",
  "total": "string",
  "total_applicable_store_credit": "string",
  "total_quantity": 0,
  "updated_at": "string",
  "user_id": 0
}

[kennyadsl]

Thanks Fabian, this is a constructive recap (except for the last point, which IMHO doesn't make sense because we are here discussing, invalidating your point).

Some notes:

• Shopify didn't discard metafields. They discarded the previous split they have with public/private (which I guess is what inspired the Spree implementation). Shopify now has a single type of metafields where all the access information are stored on the metafield itself.

This is smart because as soon as we try, it's tough to imagine putting those fields in fixed buckets. You already spotted 3 categories while describing the feature which could (not sure) have inconsistencies, for example, I don't understand the access level of the Customer Meta Data. Is it public or private?

• If it's private, does it mean that users cannot see their line item customizations after creating them?
• If it's public, can users access line item 3rd party system references?

[fthobe]

Hey,

thank you for your reply. Some clarifications, where I use the words "In my opinion" it is solely to underline that personally have a POV that can also be wrong and corrected.

1. Public in my opinion is anything done, perceived, or existing in open view, (not necessarily editable) to the general public and nothing else therefor calling any customer line item data public is for me by default wrong (I am willing to educated). Looking at the definition of public in the edge guide the documentation seems to concur with me here. If registered user access is something that Spree / Solidus defined as public that is not covered by my personal definition of the word, but I can adjust the proposal to that.
2. Customer Meta Data should only be available to the user making the order and superior administrative roles. Therefore a customer meta data item such as a PO-Number should be available with the same authorisations as other elements inside that order and not be available to the general public and therefor be private.
3. Administrative Meta Data is probably by all understanding similar to private meta data as envisioned by Spree.
4. I personally believe that catalogue meta data would be the only truly public metadata allowing a bare read access to the public.
5. The possibility to add customer metadata should by logic be reserved to line items and orders as products are in immutable as any mutation would cause it to be a different product and in my opinion it's not a job of the platform to create outcome variability on products outside of actions created by administrative roles.
6. Hence the above it became clear to me that catalogue data needed a different approach. Considering that catalogue data is public data I would not allow to mix it as that approach is prone to errors. In retrospective Public Catalogue Data is probably easiest to obtain also from properties, but I believe a placement in properties would result in additional complexities in front-end design as it would be prone to create precedents requiring operators to call properties individually in the front-end, which is something I find in complex scenarios also prone to errors. The alternative here would be fully custom fields and I can settle for that.

Some additional notes regarding maintenance of key value pairs:

• It is in my opinion the operators requirement to keep the key value pairs unique where needed, duplicates should be rejected only on per call bases: a single resource should not have more than one key with the same name and the system should validate for that
• I do believe that an issue of inconsistency is only posed if an external system does not consistently manage the key value pairs, in itself I do not see how an inconsistency can be formed by this feature other than wrong implementation on frontend or api consumption
• maybe a list of systemwide key names should be blocked for usage by this feature

[fthobe]

Hey @kennyadsl, I studied your suggestion.

I find the outcome again shifts over the burden to maintain custom fields in the code not providing any significant difference compared to a solution where a store operator customizes the fields by adding custom code. I am willing to reason with any suggestion that creates an outcome where fields can be random (meaning the key and value are not defined in code) and editable in two sets:

• one for users / customers & admins on transactions
• one reserved to admins for transactions

and in addition, yes, in a perfect world also one set for catalogue data publicly readable to the public editable only by admins.

Before I dedicate myself to a more dettaigled response to your suggestion and I can't stress this enough:

• any implementation of roles has our full support;
• any implementation that digests fully variable input / outbut on the desired resources with reasonable safety measures;

is acceptable for us.

You solution does not create an outcome significantly different to manually hack in a field into the code. It renders the solution admittedly less static than before, but it does not significantly change the dynamic. I am really interested to find a solution here, but TBH all you offer is the same amount of outcome variability that already pains this platform wrapped up in a different flavour of storing data without targeting the real problem: if any shop operator needs to create every time a custom field or implementation updating is a pain as we saw with taxons and brands. I do not want to have this level of maintenance and to be honest looking at the ecosystem it's exactly this we are currently struggling with across every extension and feature we touch. This doesn't make your solution technically inviable in general, it makes it just not adapt to problem we are trying to solve.

I personally, with my very limited knowledge, do not believe that the burden of maintaining two additional columns per transaction resource does create any significant additional burden compared to variety of different implementations of solidus that are currently making up the installation base. I personally believe that a feature like metadata for customers and admins could render future installations of solidus easy to maintain and update as they they largely remove the need for custom fields and customisations, which is the goal we are personally aiming for (and that in all transparency aligns with our overall business goals). I extend my (I really mean it) warmest invitation to you to help us get out goal achieved in a collaborative way or simply help us reach the opinion that a for us desired outcome is not possible. It's not blackmailing you to say that we suspend contributions, it's a business decision as you probably make others with your organisation as well. Given that the entire discussion across issue and PR is going on for almost a month now I am looking for a solution within a reasonable amount of time.