From 38076f6f5234c8ac45863441f2613f51da386d16 Mon Sep 17 00:00:00 2001 From: Fanit Kolchina Date: Thu, 12 Sep 2024 18:33:33 -0400 Subject: [PATCH] Add links Signed-off-by: Fanit Kolchina --- ...4-09-12-opensearch-project-roadmap-2024-2025.md | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/_posts/2024-09-12-opensearch-project-roadmap-2024-2025.md b/_posts/2024-09-12-opensearch-project-roadmap-2024-2025.md index e98159e14d..74c7160104 100644 --- a/_posts/2024-09-12-opensearch-project-roadmap-2024-2025.md +++ b/_posts/2024-09-12-opensearch-project-roadmap-2024-2025.md @@ -11,7 +11,7 @@ categories: - community-updates excerpt: OpenSearch is a rapidly growing open-source product suite comprising a search engine, an ingestion system, language clients, and a user interface for analytics. OpenSearch contributors and maintainers are innovating in all these areas at a fast pace. To steer the project's development effectively, we have revamped the project roadmap to provide better transparency into both short- and long-term enhancements. In this blog post, we are excited to share the new theme-based, community-driven OpenSearch Project Roadmap for 2024–2025. featured_blog_post: true -additional_author_info: We sincerely appreciate the contributions to this blog from Yupeng Fu (Uber), Karthikeyan Ramasamy (Uber), Shuyi Z. (Uber), Shubham Gupta (Uber), George Luong (Slack/Salesforce), Austin Lee (Aryn), Andriy Redko (Aiven), Yuncheng Lu (ByteDance), Tianjin Li (Cohere), and AWS contributors (Anandhi Bumstead, Carl Meadows, Michael Froh, Shweta Thareja, Saurabh Singh, Kunal Khatua, Andrew Ross, Sean Zheng, Anirudha Jadhav, Gopala Krishna Ambareesh, Yan Zeng, Huan Jiang, Vamshi Vijay Nakkirtha, Craig Perkins, Peter Zhu, Joshua Bright, David Venable, Bukhtawar Khan, Prabhakar Sithanandam, Ranjith Ramachandra, Sorabh Hamirwasia, Charlie Yang, Rohit Wali, Peng Huo, and Fanit Kolchina). +additional_author_info: We sincerely appreciate the contributions to this blog from Karthikeyan Ramasamy (Uber), Shuyi Z. (Uber), Shubham Gupta (Uber), George Luong (Slack/Salesforce), Austin Lee (Aryn), Andriy Redko (Aiven), Yuncheng Lu (ByteDance), Tianjin Li (Cohere), and AWS contributors (Anandhi Bumstead, Carl Meadows, Michael Froh, Shweta Thareja, Saurabh Singh, Kunal Khatua, Andrew Ross, Sean Zheng, Anirudha Jadhav, Gopala Krishna Ambareesh, Yan Zeng, Huan Jiang, Vamshi Vijay Nakkirtha, Craig Perkins, Peter Zhu, Joshua Bright, David Venable, Bukhtawar Khan, Prabhakar Sithanandam, Ranjith Ramachandra, Sorabh Hamirwasia, Charlie Yang, Rohit Wali, Peng Huo, and Fanit Kolchina). meta_keywords: OpenSearch roadmap, open source search, vector database, generative AI, observability, log analytics, security analytics, cloud native architecture, OpenSearch performance optimization meta_description: Explore the OpenSearch Project's roadmap for 2024-2025 and learn how the community is driving the project forward to make it the preferred open-source solution for search, analytics, and generative AI applications. --- @@ -35,13 +35,21 @@ In this blog post, we will outline the OpenSearch roadmap for 2024–2025, focus In the rest of this post, we will first [summarize the key innovation areas](#roadmap-summary) in the context of the roadmap themes. For readers interested in a comprehensive understanding, we have a [section dedicated to each theme](#roadmap-details) containing information about key innovations and links to the relevant GitHub RFCs/METAs for the features. + + ## Roadmap summary As a technology, OpenSearch innovates in three main areas: search, streaming data, and vectors. Search use cases employ lexical and semantic means to match end user queries to the catalog of information, stored in indexes, that drives your application. _Streaming data_ includes a wide range of real-time data types, such as raw log data, observability trace data, security event data, metric data, and other event data like Internet of Things (IoT) events. Vector data includes the outputs of embedding-generating large language models (LLMs), vectors produced by machine learning (ML) models, and encodings of media like audio and video. -OpenSearch's roadmap is aligned vertically in some cases and horizontally in others, depending on the workloads it supports. Features relevant to **search workloads** are described in [theme 1](#roadmap-theme-1-vector-database-and--generative-ai) and [theme 2](#roadmap-theme-2-search). Features relevant to **vector workloads** are described in [theme 1](#roadmap-theme-1-vector-database-and--generative-ai). Features relevant to **streaming data workloads** are described in [theme 4](#_roadmap-theme-4-observability-log-analytics-and-security-analytics). Features relevant to **all three workload types** are described in [theme 3](#roadmap-theme-3-ease-of-use), and [themes 5--9](#roadmap-theme-5-cost-performance-and-scalability). +OpenSearch's roadmap is aligned vertically in some cases and horizontally in others, depending on the workloads it supports. Features relevant to **search workloads** are described in [Theme 1](#roadmap-theme-1-vector-database-and-generative-ai) and [Theme 2](#roadmap-theme-2-search). Features relevant to **vector workloads** are described in [Theme 1](#roadmap-theme-1-vector-database-and-generative-ai). Features relevant to **streaming data workloads** are described in [Theme 4](#roadmap-theme-4-observability-log-analytics-and-security-analytics). Features relevant to **all three workload types** are described in [Theme 3](#roadmap-theme-3-ease-of-use), and [Themes 5--9](#roadmap-theme-5-cost-performance-and-scalability). -**Theme 1 (Vector Database and Generative AI)** is centered on price performance and ease of use for vector workloads, creating new features that help reduce costs through quantization, disk storage, and GPU utilization. Ease-of-use features will make it easier to get started with and use embedding vectors to improve search results. **Theme 2 (Search)** focuses on enhancing the query capabilities of core search, building a new query engine with query planning, tight integrations with Lucene innovations, improving search relevance, and searching across external data sources with Data Prepper. **Theme 3 (Ease of Use)** encompasses building a richer dashboard experience and serverless dashboards that feature simplified installation, migration, and multi-data-source support. **Theme 4 (Observability, Log Analytics, and Security Analytics)** emphasizes integrating with industry standards, such as OpenTelemetry, to unify workflows across metrics, logs, and traces; providing a richer SQL-PPL experience; positioning Discover as the main entry point for analytical workflows; improving Data Prepper for various analytics use cases; and developing well-integrated security analytics workflows. **Theme 5 (Cost, Performance, and Scalability)** includes improving core search engine performance, scaling shard management, providing context-aware templates for different workloads, moving to remote-store-backed tiered storage, and scaling cluster management. **Theme 6 (Stability, Availability, and Resiliency)** includes features involving query visibility, query resiliency, workload management, and cluster management resilience. **Theme 7 (Security)** centers on providing constructs that are secure by default and adopting a streamlined plugin security model as the plugin ecosystem grows. **Theme 8 (Modular Architecture)** involves modularizing the OpenSearch codebase to suit different deployments and moving to a decoupled, service-oriented architecture. **Theme 9 (Releases and Project Health)** dives into initiatives for faster automated releases, with streamlined continuous integration/continuous delivery (CI/CD) and metrics dashboards to measure community health and operations. +[**Theme 1 (Vector Database and Generative AI)**](#roadmap-theme-1-vector-database-and-generative-ai) is centered on price performance and ease of use for vector workloads, creating new features that help reduce costs through quantization, disk storage, and GPU utilization. Ease-of-use features will make it easier to get started with and use embedding vectors to improve search results. [**Theme 2 (Search)**](#roadmap-theme-2-search) focuses on enhancing the query capabilities of core search, building a new query engine with query planning, tight integrations with Lucene innovations, improving search relevance, and searching across external data sources with Data Prepper. [**Theme 3 (Ease of Use)**](#roadmap-theme-3-ease-of-use) encompasses building a richer dashboard experience and serverless dashboards that feature simplified installation, migration, and multi-data-source support. [**Theme 4 (Observability, Log Analytics, and Security Analytics)**](#roadmap-theme-4-observability-log-analytics-and-security-analytics) emphasizes integrating with industry standards, such as OpenTelemetry, to unify workflows across metrics, logs, and traces; providing a richer SQL-PPL experience; positioning Discover as the main entry point for analytical workflows; improving Data Prepper for various analytics use cases; and developing well-integrated security analytics workflows. [**Theme 5 (Cost, Performance, and Scalability)**](#roadmap-theme-5-cost-performance-and-scalability) includes improving core search engine performance, scaling shard management, providing context-aware templates for different workloads, moving to remote-store-backed tiered storage, and scaling cluster management. [**Theme 6 (Stability, Availability, and Resiliency)**](#roadmap-theme-6-stability-availability-and-resiliency) includes features involving query visibility, query resiliency, workload management, and cluster management resilience. [**Theme 7 (Security)**](#roadmap-theme-7-security) centers on providing constructs that are secure by default and adopting a streamlined plugin security model as the plugin ecosystem grows. [**Theme 8 (Modular Architecture)**](#roadmap-theme-8-modular-architecture) involves modularizing the OpenSearch codebase to suit different deployments and moving to a decoupled, service-oriented architecture. [**Theme 9 (Releases and Project Health)**](#roadmap-theme-9-releases-and-project-health) dives into initiatives for faster automated releases, with streamlined continuous integration/continuous delivery (CI/CD) and metrics dashboards to measure community health and operations. ## Roadmap details