Skip to content

Commit 07c1182

Browse files
Housekeeping (#379)
1 parent 3ce6e16 commit 07c1182

File tree

7 files changed

+78
-53
lines changed

7 files changed

+78
-53
lines changed

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ Things we are currently working on:
1313
- [x] ~~App: Configure embedding providers (PR [#224](https://github.com/MindWorkAI/AI-Studio/pull/224))~~
1414
- [x] ~~App: Implement an [ERI](https://github.com/MindWorkAI/ERI) server coding assistant (PR [#231](https://github.com/MindWorkAI/AI-Studio/pull/231))~~
1515
- [x] ~~App: Management of data sources (local & external data via [ERI](https://github.com/MindWorkAI/ERI)) (PR [#259](https://github.com/MindWorkAI/AI-Studio/pull/259), [#273](https://github.com/MindWorkAI/AI-Studio/pull/273))~~
16-
- [ ] Runtime: Extract data from txt / md / pdf / docx / xlsx files
16+
- [x] ~~Runtime: Extract data from txt / md / pdf / docx / xlsx files (PR [#374](https://github.com/MindWorkAI/AI-Studio/pull/374))~~
1717
- [ ] (*Optional*) Runtime: Implement internal embedding provider through [fastembed-rs](https://github.com/Anush008/fastembed-rs)
1818
- [ ] App: Implement external embedding providers
1919
- [ ] App: Implement the process to vectorize one local file using embeddings

app/MindWork AI Studio/Pages/About.razor

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -102,12 +102,17 @@
102102
<ThirdPartyComponent Name="serde" Developer="Erick Tryzelaar, David Tolnay & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/serde-rs/serde/blob/master/LICENSE-MIT" RepositoryUrl="https://github.com/serde-rs/serde" UseCase="Now we have multiple systems, some developed in .NET and others in Rust. The data format JSON is responsible for translating data between both worlds (called data serialization and deserialization). Serde takes on this task in the Rust world. The counterpart in the .NET world is an integral part of .NET and is located in System.Text.Json."/>
103103
<ThirdPartyComponent Name="keyring" Developer="Walther Chen, Daniel Brotsky & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/hwchen/keyring-rs/blob/master/LICENSE-MIT" RepositoryUrl="https://github.com/hwchen/keyring-rs" UseCase="In order to use any LLM, each user must store their so-called token for each LLM provider. This token must be kept secure, similar to a password. The safest way to do this is offered by operating systems like macOS, Windows, and Linux: They have mechanisms to store such data, if available, on special security hardware. Since this is currently not possible in .NET, we use this Rust library."/>
104104
<ThirdPartyComponent Name="arboard" Developer="Artur Kovacs, Avi Weinstock, 1Password & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/1Password/arboard/blob/master/LICENSE-MIT.txt" RepositoryUrl="https://github.com/1Password/arboard" UseCase="To be able to use the responses of the LLM in other apps, we often use the clipboard of the respective operating system. Unfortunately, in .NET there is no solution that works with all operating systems. Therefore, I have opted for this library in Rust. This way, data transfer to other apps works on every system."/>
105-
<ThirdPartyComponent Name="tokio" Developer="Alex Crichton & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/tokio-rs/tokio/blob/master/LICENSE" RepositoryUrl="https://github.com/tokio-rs/tokio" UseCase="Code in the Rust language can be specified as synchronous or asynchronous. Unlike .NET and the C# language, Rust cannot execute asynchronous code by itself. Rust requires support in the form of an executor for this. Tokio is one such executor."/>
105+
<ThirdPartyComponent Name="tokio" Developer="Alex Crichton, Carl Lerche, Alice Ryhl, Taiki Endo, Ivan Petkov, Eliza Weisman, Lucio Franco & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/tokio-rs/tokio/blob/master/LICENSE" RepositoryUrl="https://github.com/tokio-rs/tokio" UseCase="Code in the Rust language can be specified as synchronous or asynchronous. Unlike .NET and the C# language, Rust cannot execute asynchronous code by itself. Rust requires support in the form of an executor for this. Tokio is one such executor."/>
106+
<ThirdPartyComponent Name="futures" Developer="Alex Crichton, Taiki Endo, Taylor Cramer, Nemo157, Josef Brandl, Aaron Turon & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/rust-lang/futures-rs/blob/master/LICENSE-MIT" RepositoryUrl="https://github.com/rust-lang/futures-rs" UseCase="This is a library providing the foundations for asynchronous programming in Rust. It includes key trait definitions like Stream, as well as utilities like join!, select!, and various futures combinator methods which enable expressive asynchronous control flow."/>
107+
<ThirdPartyComponent Name="async-stream" Developer="Carl Lerche, Taiki Endo & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/tokio-rs/async-stream/blob/master/LICENSE" RepositoryUrl="https://github.com/tokio-rs/async-stream" UseCase="This library is used to create asynchronous streams in Rust. It allows us to work with streams of data that can be produced asynchronously, making it easier to handle events or data that arrive over time. We use this, e.g., to stream arbitrary data from the file system to the embedding system."/>
106108
<ThirdPartyComponent Name="flexi_logger" Developer="emabee & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/emabee/flexi_logger/blob/master/LICENSE-MIT" RepositoryUrl="https://github.com/emabee/flexi_logger" UseCase="This Rust library is used to output the app's messages to the terminal. This is helpful during development and troubleshooting. This feature is initially invisible; when the app is started via the terminal, the messages become visible."/>
107109
<ThirdPartyComponent Name="rand" Developer="Rust developers & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/rust-random/rand/blob/master/LICENSE-MIT" RepositoryUrl="https://github.com/rust-random/rand" UseCase="We must generate random numbers, e.g., for securing the interprocess communication between the user interface and the runtime. The rand library is great for this purpose."/>
108110
<ThirdPartyComponent Name="base64" Developer="Marshall Pierce, Alice Maz & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/marshallpierce/rust-base64/blob/master/LICENSE-MIT" RepositoryUrl="https://github.com/marshallpierce/rust-base64" UseCase="For some data transfers, we need to encode the data in base64. This Rust library is great for this purpose."/>
109111
<ThirdPartyComponent Name="Rust Crypto" Developer="Artyom Pavlov, Tony Arcieri, Brian Warner, Arthur Gautier, Vlad Filippov, Friedel Ziegelmayer, Nicolas Stalder & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/RustCrypto/traits/blob/master/cipher/LICENSE-MIT" RepositoryUrl="https://github.com/RustCrypto" UseCase="When transferring sensitive data between Rust runtime and .NET app, we encrypt the data. We use some libraries from the Rust Crypto project for this purpose: cipher, aes, cbc, pbkdf2, hmac, and sha2. We are thankful for the great work of the Rust Crypto project."/>
110112
<ThirdPartyComponent Name="rcgen" Developer="RustTLS developers, est31 & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/rustls/rcgen/blob/main/LICENSE" RepositoryUrl="https://github.com/rustls/rcgen" UseCase="For the secure communication between the user interface and the runtime, we need to create certificates. This Rust library is great for this purpose."/>
113+
<ThirdPartyComponent Name="file-format" Developer="Mickaël Malécot & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/mmalecot/file-format/blob/main/LICENSE-MIT" RepositoryUrl="https://github.com/mmalecot/file-format" UseCase="This library is used to determine the file type of a file. This is necessary, e.g., when we want to stream a file."/>
114+
<ThirdPartyComponent Name="calamine" Developer="Johann Tuffe, Joel Natividad, Eric Jolibois, Dmitriy & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/tafia/calamine/blob/master/LICENSE-MIT.md" RepositoryUrl="https://github.com/tafia/calamine" UseCase="This library is used to read Excel and OpenDocument spreadsheet files. This is necessary, e.g., for using spreadsheets as a data source for a chat."/>
115+
<ThirdPartyComponent Name="pdfium-render" Developer="Alastair Carey, Dorian Rudolph & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/ajrcarey/pdfium-render/blob/master/LICENSE.md" RepositoryUrl="https://github.com/ajrcarey/pdfium-render" UseCase="This library is used to read PDF files. This is necessary, e.g., for using PDFs as a data source for a chat."/>
111116
<ThirdPartyComponent Name="Lua-CSharp" Developer="Yusuke Nakada & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/nuskey8/Lua-CSharp/blob/main/LICENSE" RepositoryUrl="https://github.com/nuskey8/Lua-CSharp" UseCase="We use Lua as the language for plugins. Lua-CSharp lets Lua scripts communicate with AI Studio and vice versa. Thank you, Yusuke Nakada, for this great library." />
112117
<ThirdPartyComponent Name="HtmlAgilityPack" Developer="ZZZ Projects & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/zzzprojects/html-agility-pack/blob/master/LICENSE" RepositoryUrl="https://github.com/zzzprojects/html-agility-pack" UseCase="We use the HtmlAgilityPack to extract content from the web. This is necessary, e.g., when you provide a URL as input for an assistant."/>
113118
<ThirdPartyComponent Name="ReverseMarkdown" Developer="Babu Annamalai & Open Source Community" LicenseName="MIT" LicenseUrl="https://github.com/mysticmind/reversemarkdown-net/blob/master/LICENSE" RepositoryUrl="https://github.com/mysticmind/reversemarkdown-net" UseCase="This library is used to convert HTML to Markdown. This is necessary, e.g., when you provide a URL as input for an assistant."/>

app/MindWork AI Studio/Pages/Supporters.razor

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,7 @@
7979
<Supporter Name="DevNullx64" Type="SupporterType.INDIVIDUAL" URL="https://github.com/DevNullx64" Acknowledgment="Thanks Luc for your build script contribution."/>
8080
<Supporter Name="SolsticeSpectrum" Type="SupporterType.INDIVIDUAL" URL="https://github.com/SolsticeSpectrum" Acknowledgment="Thanks for your build script contribution."/>
8181
<Supporter Name="peerschuett" Type="SupporterType.INDIVIDUAL" URL="https://github.com/peerschuett" Acknowledgment="Thanks for your contributions to the project."/>
82+
<Supporter Name="nilskruthoff" Type="SupporterType.INDIVIDUAL" URL="https://github.com/nilskruthoff" Acknowledgment="Thanks, Nils, for taking the time to learn Rust and build the foundation for local retrieval."/>
8283
</MudList>
8384
</MudPaper>
8485
</MudItem>
Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,8 @@
1-
# v0.9.39, build 214 (2025-03-xx xx:xx UTC)
1+
# v0.9.39, build 214 (2025-04-xx xx:xx UTC)
22
- Added a feature flag for the plugin system. This flag is disabled by default and can be enabled inside the app settings. Please note that this feature is still in development; there are no plugins available yet.
33
- Added the Lua library we use for the plugin system to the about page.
44
- Added the plugin overview page. This page shows all installed plugins and allows you to enable or disable them. It is only available when the plugin preview feature is enabled.
55
- Added hot reloading for plugins. When any plugin is changed, the app will automatically reload the plugin without needing to restart the app.
6+
- Added an API for streaming arbitrary local files to the embedding process. Thanks Nils `nilskruthoff` for this great contribution.
67
- Fixed the preview tooltip component not showing the correct position when used inside a scrollable container.
78
- Upgraded to Rust 1.85.1

runtime/Cargo.lock

Lines changed: 49 additions & 34 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

runtime/Cargo.toml

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,10 @@ serde_json = "1.0.134"
1616
keyring = { version = "3.6.2", features = ["apple-native", "windows-native", "sync-secret-service"] }
1717
arboard = "3.4.1"
1818
tokio = { version = "1.44", features = ["rt", "rt-multi-thread", "macros", "process"] }
19-
flexi_logger = "0.29.8"
19+
tokio-stream = "0.1.17"
20+
futures = "0.3.31"
21+
async-stream = "0.3.6"
22+
flexi_logger = "0.30.0"
2023
log = { version = "0.4.26", features = ["kv"] }
2124
once_cell = "1.20.3"
2225
rocket = { version = "0.5.1", features = ["json", "tls"] }
@@ -31,11 +34,8 @@ hmac = "0.12.1"
3134
sha2 = "0.10.8"
3235
rcgen = { version = "0.13.2", features = ["pem"] }
3336
file-format = "0.26.0"
34-
calamine = "0.22"
35-
pdfium-render = "0.8.27"
36-
async-stream = "0.3"
37-
futures = "0.3"
38-
tokio-stream = "0.1"
37+
calamine = "0.26.1"
38+
pdfium-render = "0.8.29"
3939

4040
# Fixes security vulnerability downstream, where the upstream is not fixed yet:
4141
url = "2.5"

0 commit comments

Comments
 (0)