upload cpu and gpu environment.yml files

docs/00_setup/environment-cpu.yml

@@ -0,0 +1,34 @@
+name: genai-cpu
+channels:
+  - pytorch
+  - nvidia
+  - conda-forge
+  - defaults
+dependencies:
+  - python=3.11 
+  - jupyterlab 
+  - pytorch 
+  - torchvision 
+  - torchaudio 
+  - huggingface_hub 
+  - diffusers 
+  - transformers 
+  - accelerate 
+  - anthropic 
+  - openai 
+  - langchain 
+  - stackview 
+  - voila 
+  - pandas 
+  - torchmetrics 
+  - scikit-learn 
+  - seaborn
+  - pip
+  - pip:
+      - python-pptx 
+      - bia-bob 
+      - blablado 
+      - llama-index 
+      - google-generativeai 
+      - pygithub
+prefix: C:\Users\haase\miniconda3\envs\genai-cpu

docs/00_setup/environment-gpu.yml

@@ -0,0 +1,36 @@
+name: genai-gpu
+channels:
+  - pytorch
+  - nvidia
+  - conda-forge
+  - defaults
+dependencies:
+  - python=3.11 
+  - jupyterlab 
+  - pytorch 
+  - torchvision 
+  - torchaudio
+  - pytorch-cuda=11.8 
+  - cudatoolkit=11.8
+  - huggingface_hub 
+  - diffusers 
+  - transformers 
+  - accelerate 
+  - anthropic 
+  - openai 
+  - langchain 
+  - stackview 
+  - voila 
+  - pandas 
+  - torchmetrics 
+  - scikit-learn 
+  - seaborn
+  - pip
+  - pip:
+      - python-pptx 
+      - bia-bob 
+      - blablado 
+      - llama-index 
+      - google-generativeai 
+      - pygithub
+prefix: C:\Users\haase\miniconda3\envs\genai-gpu

docs/15_endpoint_apis/30_huggingface.ipynb

@@ -0,0 +1,133 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "53269b57-8207-4e44-89aa-f2b108115430",
+   "metadata": {},
+   "source": [
+    "# Huggingface API\n",
+    "\n",
+    "The [Huggingface](https://huggingface.co) API make open source/weight models available locally. As the model is loaded into memory, we can reuse it, and do not have to reload it entirely. This might be beneficial in scenarios where we want to prompt the same model many times."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "66ee90b4-389f-4c88-8688-edd3a36a8cb1",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def prompt_hf(request, model=\"meta-llama/Meta-Llama-3.1-8B\"):\n",
+    "    global prompt_hf\n",
+    "    import transformers\n",
+    "    import torch\n",
+    "    \n",
+    "    if prompt_hf._pipeline is None:    \n",
+    "        prompt_hf._pipeline = transformers.pipeline(\n",
+    "            \"text-generation\", model=model, model_kwargs={\"torch_dtype\": torch.bfloat16}, device_map=\"auto\"\n",
+    "        )\n",
+    "    \n",
+    "    return prompt_hf._pipeline(\"Hey how are you doing today?\")\n",
+    "prompt_hf._pipeline = None"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d8e0b317-febd-4da0-a1bc-3440b5e18855",
+   "metadata": {},
+   "source": [
+    "We can then submit a prompt to the LLM like this:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "460e9067-63b9-4d11-97a0-2d97bbf5b529",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "C:\\Users\\rober\\miniconda3\\envs\\genai-cpu\\Lib\\site-packages\\torch\\cuda\\__init__.py:843: UserWarning: CUDA initialization: The NVIDIA driver on your system is too old (found version 11070). Please update your GPU driver by downloading and installing a new version from the URL: http://www.nvidia.com/Download/index.aspx Alternatively, go to: https://pytorch.org to install a PyTorch version that has been compiled with your version of the CUDA driver. (Triggered internally at C:\\cb\\pytorch_1000000000000\\work\\c10\\cuda\\CUDAFunctions.cpp:108.)\n",
+      "  r = torch._C._cuda_getDeviceCount() if nvml_count < 0 else nvml_count\n"
+     ]
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "75430d1a95914bf9930bd8238055a367",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Some parameters are on the meta device device because they were offloaded to the cpu and disk.\n",
+      "Setting `pad_token_id` to `eos_token_id`:128001 for open-end generation.\n",
+      "C:\\Users\\rober\\miniconda3\\envs\\genai-cpu\\Lib\\site-packages\\transformers\\generation\\utils.py:1259: UserWarning: Using the model-agnostic default `max_length` (=20) to control the generation length. We recommend setting `max_new_tokens` to control the maximum length of the generation.\n",
+      "  warnings.warn(\n"
+     ]
+    }
+   ],
+   "source": [
+    "prompt_hf(\"What is the capital of France?\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "4000bfbb-b075-4d8c-b9ce-f356383cd139",
+   "metadata": {},
+   "source": [
+    "As the model is kept loaded in memory, a second call might be faster, not showing the model-loading output:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "fd82b669-d8b5-454d-82bf-86ff1fc65d71",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "prompt_hf(\"What is the capital of the Czech Republic?\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "190b3090-e73c-48df-abf8-33b096f95a58",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}