Skip to content

Change rbx_reflection_database to support loading from FS#376

Merged
Dekkonot merged 25 commits intorojo-rbx:masterfrom
Dekkonot:dynamic-reflection-pog
Aug 8, 2025
Merged

Change rbx_reflection_database to support loading from FS#376
Dekkonot merged 25 commits intorojo-rbx:masterfrom
Dekkonot:dynamic-reflection-pog

Conversation

@Dekkonot
Copy link
Copy Markdown
Member

As part of the effort to get the reflection database loadable locally, it's probably a good idea to have a central place that decides locations for the entire ecosystem. This will prevent consumers like Lune and Rojo from having to re-implement this logic.

The obvious choice for this, in my opinion, is rbx_reflection_database. It's a requirement for everything that uses the reflection database already, and its public API can be extended to take things from the file system into account without any issues. I've implemented it and documented it thoroughly in the README for rbx_reflection_database.

However, it also sacrifices the 'purity' of the module because it fundamentally alters its purpose. I'm okay with that, but I understand not everyone will be. Please be honest with your thoughts.

I know for sure this works on Windows and Linux. I'm assuming it will also work on MacOS, but I don't own a machine I can use for testing. I don't know how to test this in CI, so we'll just have to be mindful of changes.

Specifically, databases are now loaded from a local directory
or from an environmental variable
if either exist.
Copy link
Copy Markdown
Member

@Corecii Corecii left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor comments and future planning concerns

Comment thread rbx_reflection_database/src/lib.rs Outdated
Comment thread rbx_reflection_database/README.md Outdated
Comment thread rbx_reflection_database/src/lib.rs Outdated
Comment thread rbx_reflection_database/src/lib.rs Outdated
Comment thread rbx_reflection_database/src/lib.rs Outdated
@Dekkonot
Copy link
Copy Markdown
Member Author

Allowing the database to be replaced once it's initialized is complicated because it involves mutating a static variable, so for the time being I'm going to ignore it. Tools like Rojo and Lune can work out solutions without that.

@Dekkonot Dekkonot requested a review from Corecii November 14, 2023 18:33
Copy link
Copy Markdown
Member

@kennethloeffler kennethloeffler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, I left a few comments.

RE testing - it looks like the only code paths not covered right now are here:

} else {
// Due to concerns about the local data directory existing
// on Linux, we use the home directory instead.
#[cfg(target_os = "linux")]
let mut home = dirs::home_dir()?;
#[cfg(not(target_os = "linux"))]
let mut home = dirs::data_local_dir()?;
home.push(LOCAL_DIR_NAME);
home.push("database.msgpack");
Some(home)
}

Maybe we could add a test that simply compares the result of this to the expected path for the platform, and if we're very paranoid, add some CI targets for macOS and Windows?

Comment thread rbx_reflection_database/src/lib.rs Outdated
Comment thread rbx_reflection_database/src/lib.rs Outdated
Comment thread rbx_reflection_database/src/lib.rs
@LouieK22
Copy link
Copy Markdown
Contributor

LouieK22 commented Jan 3, 2025

We should probably use a more generic top-level folder name (e.g. %localappdata%/rbx_dom on Windows, and ~/.rbx_dom on Mac/Linux) as it would give a better hint as to where these files came from. The reflection cache could be something like rbx_dom/cache/reflection.proto

This crate might come in handy for picking directories: https://crates.io/crates/directories

@Dekkonot
Copy link
Copy Markdown
Member Author

Dekkonot commented Jan 6, 2025

It's currently placed in %localappdata%/.rbx_reflection on Windows, $HOME/Library/Application Support/.rbxreflection on MacOS, and $HOME/.rbxreflection on Linux. .rbx_dom or something works too. It's not really important.

Copy link
Copy Markdown
Member

@kennethloeffler kennethloeffler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, and will work well for consumers that want to update the reflection database without shipping a new binary.

I think the current data directory choosing scheme works fine, probably don't need to add a library for our uses

Comment thread rbx_reflection_database/CHANGELOG.md Outdated
Comment thread rbx_reflection_database/src/lib.rs
@Dekkonot Dekkonot dismissed Corecii’s stale review August 8, 2025 02:42

Ken has approved

@Dekkonot Dekkonot merged commit 839284a into rojo-rbx:master Aug 8, 2025
3 checks passed
DervexDev added a commit to argon-rbx/rbx-dom that referenced this pull request Dec 9, 2025
* Add test for properties that don't serialize in rbx_binary (rojo-rbx#548)

* Make clippy shut up about `uninlined_format_args` (rojo-rbx#551)

* Migrate to new Content properties from old ContentId ones (rojo-rbx#550)

* Change rbx_reflection_database to support loading from FS (rojo-rbx#376)

* Fix elided lifetime lints new in Rust 1.89 (rojo-rbx#552)

* Migrate Content properties added since v684 (rojo-rbx#554)

- Migrates `BackpackItem.TextureId` to `TextureContent`
- Migrates `ClickDetector.CursorIcon` to `CursorIconContent`
- Migrates `DragDetector.ActivatedCursorIcon` to
`ActivatedCursorIconContent`
- Migrates `Decal.ColorMap` to `ColorMapContent` (just in case, as they
don't serialize but Roblox could change that)
- Migrates `Mouse.Icon` to `IconContent`
- Migrates `UIDragDetector.ActivatedCursorIcon` to
`ActivatedCursorIconContent`
- Migrates `UIDragDetector.CursorIcon` to `CursorIconContent`
- Migrates `UserInputService.MouseIcon` to `MouseIconContent`
- Migrates `AudioPlayer.Asset` to `AudioPlayer.AudioContent`
- Migrates `Sound.SoundId` to `Sound.AudioContent`

* Update reflection database to Roblox 691 (rojo-rbx#553)

* Update implementation of `PhysicalProperties` to include new `AcousticAbsorption` field (rojo-rbx#556)

* Implement NetAssetRef (rojo-rbx#555)

* Mark `FontWeight`, `FontStyle`, and `TerrainMaterials` as non-exhaustive (rojo-rbx#557)

* Release rbx-types v3.0.0

* Release rbx_dom_weak v4.0.0

* Release rbx_reflection v6.0.0

* Release rbx_reflection_database v2.0.0+roblox-694

* Release rbx_binary v2.0.0

* Release rbx_xml v2.0.0

* Use extend in `generate_referents` in rbx_binary (rojo-rbx#537)

* rbx_binary: Silence Unused Import Lint (rojo-rbx#535)

* Update reflection database to v697 (rojo-rbx#559)

* Release rbx_reflection_database v2.0.1+roblox-697

* rbx_binary: Interleaved Iterators (rojo-rbx#545)

* rbx_binary: No Cow For `PropInfo.default_value` (rojo-rbx#543)

* rbx_reflection: Add `DataType.ty()` (rojo-rbx#540)

* rbx_binary: Reuse Descriptors (rojo-rbx#542)

]

* Allow `WeakDom::transfer_within` destination to be the None Ref (rojo-rbx#517)

* Properly utilize Serde visitors for types (rojo-rbx#563)

* Use as_ref to avoid cloning SharedString (rojo-rbx#566)

* Change base64 module to use EncodingService's base64 methods (rojo-rbx#564)

* Fix a single comma causing stylua to be mad (rojo-rbx#570)

* rbx_types: Add `CFrame::identity()` (rojo-rbx#567)

* Add support for StyleRule `PropertiesSerialize` (rojo-rbx#571)

* rbx_binary: Index Properties Once (+37%) (rojo-rbx#544)

* Release rbx_types v3.1.0

* Release rbx_dom_weak v4.1.0

* Release rbx_reflection v6.1.0

* Release rbx_reflection_database v2.0.2+roblox-700

* Release rbx_binary v2.0.1

* Release rbx_xml v2.0.1

* Fix formatting

---------

Co-authored-by: krakow10 <krakow20@gmail.com>
Co-authored-by: Micah <git@dekkonot.com>
Co-authored-by: Ken Loeffler <kenloef@gmail.com>
Co-authored-by: Redus <92763949+Redusofficial@users.noreply.github.com>
Co-authored-by: Cameron Campbell <cameron.campbell325@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants