Improve the UX of moving "stuff" to the database

[rtrudel]

Context

When installing the eloquent package, users are presented with repository choices without clear explanations of their purposes. This is particularly challenging for users migrating from other CMS platforms (like Craft CMS, WordPress) where terminology and concepts may differ.

Current Issues

1. Repository selection during eloquent package installation lacks context
2. No clear distinction between content and structural elements
3. Insufficient guidance on environment-specific configurations
4. Missing cross-CMS terminology reference
5. Unclear content synchronization behaviors between flat-files and database

Proposed Improvements

1. Repository Documentation

• Clear explanation of each available repository type
• Use cases for each repository
• Implications of choosing specific repositories
• Database vs flat-file storage considerations

2. Content vs Structure Classification

• Clear guidelines on what constitutes content vs structure
• Best practices for managing each type
• Migration considerations from other CMS platforms

3. Content Synchronization Behaviors

• Detailed explanation of content priority rules:
  • How the system handles content existing in both flat-files and database
  • Which source takes precedence and why
  • Configuration options for priority settings if available
• Import/Export Behaviors:
  • What happens to flat-files after database import
  • What happens to database tables after flat-file export
  • How to handle conflicts during synchronization
  • Best practices for backup before import/export operations
• Clear documentation of cleanup operations:
  • Whether files are deleted after import
  • Whether tables are truncated after export
  • How to maintain data integrity during these operations

4. Environment Management

• Guidelines for Git workflow
• Examples of .gitignore configurations for:
  • Flat-file setups
  • Database-driven setups
  • Hybrid approaches
• Best practices for managing content across different environments

5. Migration Guide

• Terminology mapping between common CMS platforms
• Common migration patterns
• Pitfalls to avoid

6. Configuration Examples

Include practical examples for common setups:

• Pure flat-file configuration
• Database-driven setup
• Hybrid approach
• Environment-specific considerations
• Content synchronization scenarios with expected outcomes

Expected Benefits

• Reduced learning curve for new users
• Better understanding of system architecture
• Improved content/structure separation
• More reliable deployment processes
• Clear understanding of content synchronization behaviors
• Safer content migration operations

Implementation Suggestions

1. Add a new section in the main documentation
2. Include visual diagrams for complex concepts
3. Provide practical examples with common use cases
4. Add a glossary of terms with CMS-specific mappings
5. Include decision trees for content priority scenarios
6. Add warning boxes for potentially destructive operations (if applicable)

[rtrudel]

If I may suggest: Adding a CLI command that would provide certain presets to simplify storage strategy management and make it easy to switch between different configurations.

The data:schema command could help manage storage strategies for both content and structure:

# Predefined storage strategy presets
php please data:schema --preset=flat-file          # Everything in files
php please data:schema --preset=database-only      # Everything in database
php please data:schema --preset=hybrid-standard    # Content in DB, structure in files

# Custom storage strategy configuration
php please data:schema --content=db --structure=file

# Granular repository configuration
php please data:schema --collections=file --entries=db --forms=db --taxonomies=file

This would:

1. Simplify the migration process
2. Make it safer to experiment with different storage configurations
3. Provide clear visibility into current storage strategy
4. Allow for granular control over repository storage
5. Include safeguards against data loss

The command could also provide helpful feedback about the implications of each change and ensure data integrity during transitions.

[duncanmcclean]

Good idea!

I think we can probably adjust the install:eloquent-driver command (which is the CLI wizard you see) to ask what you want to move:

• Everything
• Only move content (keep configs flat file)
• Choose specific repositories

^ those options would also be available when creating new sites

We can also improve the existing documentation around the Eloquent Driver to mention content vs config repos to clarify what actually gets moved.

[rtrudel]

I believe indeed that what I call structure is named config here, but we're talking about the same thing :)

[rtrudel]

Since Statamic's default mode is flat-file, it could also be interesting to have a simple way to manage .gitignore files if we wanted, for example, to keep everything without a database, while still ensuring that content remains independent between environments. This could also be kind of preset "flat-config" or something.

[duncanmcclean]

Since Statamic's default mode is flat-file, it could also be interesting to have a simple way to manage .gitignore files if we wanted, for example, to keep everything without a database, while still ensuring that content remains independent between environments. This could also be kind of preset "flat-config" or something.

That's probably a little opinionated for us to do in Core. As far as I know, most folks using flat file share the same content between sites.

However, you're more than welcome to do that in your own projects or create a starter kit that does that (although, that would be a very small kit 😄).

[rtrudel]

That's probably a little opinionated for us to do in Core. As far as I know, most folks using flat file share the same content between sites.

I understand the concern, and probably I just do not "get it", but I'm curious how a site would work when being modified by several developers working locally (so push/pull their changes without conflicts), while the site is in production and users or writers are modifying its content (ex: new articles, e-commerce orders, user comments and reviews, etc.). Why would we sync locally all the website content when only config matters on local environment? As I said, I'm just unsure to understand this bit, maybe the answer is very simple and I'm overcomplicating things...

[ryanmitchell]

That's probably a little opinionated for us to do in Core. As far as I know, most folks using flat file share the same content between sites.

I understand the concern, and probably I just do not "get it", but I'm curious how a site would work when being modified by several developers working locally (so push/pull their changes without conflicts), while the site is in production and users or writers are modifying its content (ex: new articles, e-commerce orders, user comments and reviews, etc.). Why would we sync locally all the website content when only config matters on local environment? As I said, I'm just unsure to understand this bit, maybe the answer is very simple and I'm overcomplicating things...

We often pull content down so we are working from the latest client content, especially when developing new functionality. You get around conflicts by using a branch other than main/master.