Include RAGAS evaluation example (#23)

* add ragas example

.github/workflows/test.yml

@@ -55,4 +55,4 @@ jobs:
       #   AZURE_OPENAI_DEPLOYMENT_NAME: ${{secrets.AZURE_OPENAI_DEPLOYMENT_NAME}}
       #   OPENAI_API_VERSION: ${{secrets.OPENAI_API_VERSION}}
       run: |
-        pytest --nbval-lax python-recipes/RAG/00_intro_redispy.ipynb python-recipes/RAG/01_redisvl.ipynb python-recipes/RAG/02_langchain.ipynb python-recipes/RAG/03_llamaindex.ipynb python-recipes/RAG/04_advanced_redisvl.ipynb
+        pytest --nbval-lax python-recipes/RAG/00_intro_redispy.ipynb python-recipes/RAG/01_redisvl.ipynb python-recipes/RAG/02_langchain.ipynb python-recipes/RAG/03_llamaindex.ipynb python-recipes/RAG/04_advanced_redisvl.ipynb python-recipes/RAG/06_ragas_evaluation.ipynb

python-recipes/RAG/resources/testset.csv

@@ -0,0 +1,7 @@
+question,contexts,ground_truth,evolution_type,metadata,episode_done
+What potential consequences could securities class action litigation have on a company's operations and reputation?,"[""Our Class B Common Stock is traded publicly, and at any given time various securities analysts follow our financial results and issue reports on us. These reports include information about our historical financial results as well as analysts' opinions of our future performance, which may, in part, be based upon any guidance we have provided. Analysts' estimates are often different from our estimates or expectations. If our operating results are below the estimates or expectations of public market analysts and investors, our stock price could decline. In the past, securities class action litigation has been brought against NIKE and other companies following a decline in the market price of their securities. If our stock price is volatile for any reason, we may become involved in this type of litigation in the future. Any litigation could result in reputational damage, substantial costs and a diversion of management's attention and resources needed to successfully run our business.\n\n2023 FORM 10-K 23\n\nTable of Contents\n\nITEM 1B. UNRESOLVED STAFF COMMENTS\n\nNone.\n\nITEM 2. PROPERTIES\n\nThe following is a summary of principal properties owned or leased by NIKE:\n\nThe NIKE World Campus, owned by NIKE and located near Beaverton, Oregon, USA, is an approximately 400-acre site consisting of over 40 buildings which, together with adjacent leased properties, functions as our world headquarters and is occupied by approximately 11,400 employees engaged in management, research, design, development, marketing, finance and other administrative functions serving nearly all of our segments. We lease a similar, but smaller, administrative facility in Hilversum, the Netherlands, which serves as the headquarters for our Europe, Middle East & Africa geography and management of certain brand functions for our non-U.S. operations. We also lease an office complex in Shanghai, China, our headquarters for our Greater China geography, occupied by employees focused on implementing our wholesale, NIKE Direct and merchandising strategies in the region, among other functions.""]","Any litigation could result in reputational damage, substantial costs and a diversion of management's attention and resources needed to successfully run our business.",simple,[{'source': 'resources/nke-10k-2023.pdf'}],True
+How do currency exchange rates impact the financial results of the company?,"[""Economic factors beyond our control, and changes in the global economic environment, including fluctuations in inflation and currency exchange rates, could result in lower revenues, higher costs and decreased margins and earnings.\n\nA majority of our products are manufactured and sold outside of the United States, and we conduct purchase and sale transactions in various currencies, which creates exposure to the volatility of global economic conditions, including fluctuations in inflation and foreign currency exchange rates. Central banks may deploy various strategies to combat inflation, including increasing interest rates, which may impact our borrowing costs. Additionally, there has been, and may continue to be, volatility in currency exchange rates that impact the U.S. Dollar value relative to other international currencies. Our international revenues and expenses generally are derived from sales and operations in foreign currencies, and these revenues and expenses are affected by currency fluctuations, specifically amounts recorded in foreign currencies and translated into U.S. Dollars for consolidated financial reporting, as weakening of foreign currencies relative to the U.S. Dollar adversely affects the U.S. Dollar value of the Company's foreign currency-denominated sales and earnings. Currency exchange rate fluctuations could also disrupt the business of the independent manufacturers that produce our products by making their purchases of raw materials more expensive and more difficult to finance. Foreign currency fluctuations have adversely affected and could continue to have an adverse effect on our results of operations and financial condition.\n\nWe hedge certain foreign currency exposures to lessen and delay, but not to completely eliminate, the effects of foreign currency fluctuations on our financial results. Since the hedging activities are designed to lessen volatility, they not only reduce the negative impact of a stronger U.S. Dollar or other trading currency, but they also reduce the positive impact of a weaker U.S. Dollar or other trading currency. Our future financial results have in the past been and could in the future be significantly affected by the value of the U.S. Dollar in relation to the foreign currencies in which we conduct business. The degree to which our financial results are affected for any given time period will depend in part upon our hedging activities.""]",Currency exchange rate fluctuations could disrupt the business of the independent manufacturers that produce our products by making their purchases of raw materials more expensive and more difficult to finance. Foreign currency fluctuations have adversely affected and could continue to have an adverse effect on our results of operations and financial condition.,simple,[{'source': 'resources/nke-10k-2023.pdf'}],True
+Why are cost-effective investments considered essential for business growth and profitability?,"['From time to time, we may invest in technology, business infrastructure, new businesses or capabilities, product offering and manufacturing innovation and expansion of existing businesses, such as our NIKE Direct operations, which require substantial cash investments and management attention. We believe cost-effective investments are essential to business growth and profitability; however, significant investments are subject to typical risks and uncertainties inherent in developing a new business or expanding an existing business. The failure of any significant investment to provide expected returns or profitability could have a material adverse effect on our financial results and divert management attention from more profitable business operations. See also ""Our NIKE Direct operations have required and will continue to require a substantial investment and commitment of resources and are subject to numerous risks and uncertainties.""\n\nThe sale of a large number of shares of common stock by our principal shareholder could depress the market price of our common stock.\n\nAs of June 30, 2023, Swoosh, LLC beneficially owned approximately 77% of our Class A Common Stock. If, on June 30, 2023, all of these shares were converted into Class B Common Stock, Swoosh, LLC\'s commensurate ownership percentage of our Class B Common Stock would be approximately 16%. The shares are available for resale, subject to the requirements of the U.S. securities laws and the terms of the limited liability company agreement governing Swoosh, LLC. The sale or prospect of a sale of a substantial number of these shares could have an adverse effect on the market price of our common stock. Swoosh, LLC was formed by Philip H. Knight, our Chairman Emeritus, to hold the majority of his shares of Class A Common Stock. Mr. Knight does not have voting rights with respect to Swoosh, LLC, although Travis Knight, his son and a NIKE director, has a significant role in the management of the Class A Common Stock owned by Swoosh, LLC.\n\nChanges in our credit ratings or macroeconomic conditions may affect our liquidity, increasing borrowing costs and limiting our financing options.']","Cost-effective investments are considered essential for business growth and profitability because they contribute to the development of new businesses, expansion of existing businesses, and the improvement of product offerings and manufacturing innovation. These investments help drive growth and increase profitability. However, significant investments also come with risks and uncertainties.",simple,[{'source': 'resources/nke-10k-2023.pdf'}],True
+What changes were made to the U.S. corporate income tax system by the Inflation Reduction Act of 2022?,"['FISCAL 2023 COMPARED TO FISCAL 2022\n\nOther (income) expense, net increased from $181 million of other income, net in fiscal 2022 to $280 million in the current fiscal year, primarily due to a net favorable change in foreign currency conversion gains and losses, including hedges, and the one-time charge related to the deconsolidation of our Russian operations recognized in the prior year. This increase was partially offset by net unfavorable activity related to our strategic distributor partnership transition within APLA, including the loss recognized upon the completion of the sale of our entities in Argentina and Uruguay to a third-party distributor in the second quarter of fiscal 2023.\n\nFor more information related to our distributor partnership transition within APLA, see Note 18 — Acquisitions and Divestitures within the accompanying Notes to the Consolidated Financial Statements.\n\nWe estimate the combination of the translation of foreign currency-denominated profits from our international businesses, and the year-over-year change in foreign currency-related gains and losses included in Other (income) expense, net had an unfavorable impact on our Income before income taxes of $1,023 million for fiscal 2023. INCOME TAXES\n\nFISCAL 2023\n\nFISCAL 2022\n\n% CHANGE\n\nFISCAL 2021\n\n% CHANGE\n\nEffective tax rate\n\n18.2 %\n\n9.1 %\n\n910 bps\n\n14.0 %\n\n(490) bps\n\nFISCAL 2023 COMPARED TO FISCAL 2022\n\nOur effective tax rate was 18.2% for fiscal 2023, compared to 9.1% for fiscal 2022, primarily due to decreased benefits from stock-based compensation and a non-cash, one-time benefit in the prior year related to the onshoring of certain non-U.S. intangible property ownership rights.\n\nOn August 16, 2022, the U.S. government enacted the Inflation Reduction Act of 2022 that includes, among other provisions, changes to the U.S. corporate income tax system, including a fifteen percent minimum tax based on ""adjusted financial statement income,"" which is effective for NIKE beginning June 1, 2023. Based on our current analysis of the provisions, we do not expect these tax law changes to have a material impact on our financial statements; however, we will continue to evaluate their impact as further information becomes available.\n\n2023 FORM 10-K 35\n\nTable of Contents\n\nOPERATING SEGMENTS']","The Inflation Reduction Act of 2022 made changes to the U.S. corporate income tax system, including the implementation of a fifteen percent minimum tax based on 'adjusted financial statement income', effective for NIKE beginning June 1, 2023.",simple,[{'source': 'resources/nke-10k-2023.pdf'}],True
+"How does master netting arrangements impact recording of assets and liabilities for derivative financial instruments, and how are cash collateral amounts for credit-related contingent features accounted for?","[""The Company records the assets and liabilities of its derivative financial instruments on a gross basis on the Consolidated Balance Sheets. The Company's derivative financial instruments are subject to master netting arrangements that allow for the offset of assets and liabilities in the event of default or early termination of the contract. Any amounts of cash collateral received related to these instruments associated with the Company's credit-related contingent features are recorded in Cash and equivalents and Accrued liabilities, the latter of which would further offset against the Company's derivative asset balance. Any amounts of cash collateral posted related to these instruments associated with the Company's credit-related contingent features are recorded in Prepaid expenses and other current assets, which would further offset against the Company's derivative liability balance. Cash collateral received or posted related to the Company's credit-related contingent features is presented in the Cash provided by operations component of the Consolidated Statements of Cash Flows. The Company does not recognize amounts of non-cash collateral received, such as securities, on the Consolidated Balance Sheets. For further information related to credit risk, refer to Note 12 — Risk Management and Derivatives.\n\n2023 FORM 10-K 68\n\nTable of Contents\n\nThe following tables present information about the Company's derivative assets and liabilities measured at fair value on a recurring basis and indicate the level in the fair value hierarchy in which the Company classifies the fair value measurement:\n\nMAY 31, 2023\n\nDERIVATIVE ASSETS\n\nDERIVATIVE LIABILITIES\n\n(Dollars in millions)\n\nASSETS AT FAIR VALUE\n\nOTHER CURRENT ASSETS\n\nOTHER LONG- TERM ASSETS\n\nLIABILITIES AT FAIR VALUE\n\nACCRUED LIABILITIES\n\nOTHER LONG- TERM LIABILITIES\n\nLevel 2:\n\nForeign exchange forwards and options\n\n(1)\n\n$\n\n557 $\n\n493 $\n\n64 $\n\n180 $\n\n128 $\n\n52\n\n(1)\n\nIf the foreign exchange derivative instruments had been netted on the Consolidated Balance Sheets, the asset and liability positions each would have been reduced by $178 million as of May 31, 2023. As of that date, the Company received $36 million of cash collateral from various counterparties related to foreign exchange derivative instruments. No amount of collateral was posted on the derivative liability balance as of May 31, 2023.\n\nMAY 31, 2022\n\nDERIVATIVE ASSETS\n\nDERIVATIVE LIABILITIES\n\n(Dollars in millions)\n\nASSETS AT FAIR VALUE""]","The Company's derivative financial instruments are subject to master netting arrangements that allow for the offset of assets and liabilities in the event of default or early termination of the contract. Cash collateral received related to these instruments associated with the Company's credit-related contingent features are recorded in Cash and equivalents and Accrued liabilities, the latter of which would further offset against the Company's derivative asset balance. Cash collateral posted related to these instruments associated with the Company's credit-related contingent features are recorded in Prepaid expenses and other current assets, which would further offset against the Company's derivative liability balance.",multi_context,[{'source': 'resources/nke-10k-2023.pdf'}],True
+"How are gains and losses determined for short-term investments, specifically available-for-sale debt securities, according to the Company's accounting policies?","[""CASH AND EQUIVALENTS Cash and equivalents represent cash and short-term, highly liquid investments, that are both readily convertible to known amounts of cash and so near their maturity they present insignificant risk of changes in value because of changes in interest rates, with maturities three months or less at the date of purchase.\n\n2023 FORM 10-K 62\n\nTable of Contents\n\nSHORT-TERM INVESTMENTS\n\nShort-term investments consist of highly liquid investments with maturities over three months at the date of purchase. At May 31, 2023 and 2022, Short-term investments consisted of available-for-sale debt securities, which are recorded at fair value with unrealized gains and losses reported, net of tax, in Accumulated other comprehensive income (loss), unless unrealized losses are determined to be unrecoverable. Realized gains and losses on the sale of securities are determined by specific identification. The Company considers all available-for-sale debt securities, including those with maturity dates beyond 12 months, as available to support current operational liquidity needs and, therefore, classifies all securities with maturity dates beyond three months at the date of purchase as current assets within Short-term investments on the Consolidated Balance Sheets.\n\nRefer to Note 4 — Fair Value Measurements for more information on the Company's Short-term investments.\n\nALLOWANCE FOR UNCOLLECTIBLE ACCOUNTS RECEIVABLE Accounts receivable, net consist primarily of amounts due from customers. The Company makes ongoing estimates relating to the collectability of its accounts receivable and maintains an allowance for expected losses resulting from the inability of its customers to make required payments. In addition to judgments about the creditworthiness of significant customers based on ongoing credit evaluations, the Company considers historical levels of credit losses, as well as macroeconomic and industry trends to determine the amount of the allowance. The allowance for uncollectible accounts receivable was $35 million and $34 million as of May 31, 2023 and 2022, respectively.\n\nINVENTORY VALUATION""]",Realized gains and losses on the sale of securities are determined by specific identification.,multi_context,[{'source': 'resources/nke-10k-2023.pdf'}],True

requirements.txt

@@ -4,6 +4,7 @@ openai
 tiktoken
 langchain
 langchain-community
+langchain-openai
 llama-index
 llama-index-vector-stores-redis
 llama-index-embeddings-cohere
@@ -16,3 +17,6 @@ nbval
 redis
 redisvl
 pytest
+ragas
+datasets
+