Skip to content

Commit

Permalink
Merge remote-tracking branch 'upstream/master' into add_keyword_only_…
Browse files Browse the repository at this point in the history
…actors
  • Loading branch information
WassCodeur committed Jul 11, 2024
2 parents c9b166b + 56a6177 commit f873d5d
Show file tree
Hide file tree
Showing 8 changed files with 505 additions and 4 deletions.
6 changes: 3 additions & 3 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: [3.8, 3.9, '3.10', 3.11]
python-version: [3.9, '3.10', 3.11, 3.12]
os: [ubuntu-latest, macos-latest, windows-latest]
platform: [x64]
install-type: [pip, ] # conda]
Expand All @@ -34,13 +34,13 @@ jobs:
use-pre: [false]
include:
- os: macos-latest # ubuntu-latest
python-version: '3.10'
python-version: 3.11
install-type: pip
depends: OPTIONAL_DEPS
coverage: true
use-pre: false
- os: ubuntu-latest
python-version: '3.10'
python-version: 3.12
install-type: pip
depends: OPTIONAL_DEPS
coverage: false
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
74 changes: 74 additions & 0 deletions docs/source/posts/2024/2024-06-26-week3-wachiou-bouraima.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
WEEK 3: Refinements and Further Enhancements
============================================

.. post:: June 26, 2024
:author: Wachiou BOURAIMA
:tags: google
:category: gsoc

Hello everyone,
---------------

Welcome to the fourth week of my Google Summer of Code (GSoC) 2024 journey!
This week I've been delving into the technical aspects of my project,
focusing on the consistent application of the ``warn_on_args_to_kwargs`` decorator and the initial implementation of lazy loading.


Consistent application of ``warn_on_args_to_kwargs``
----------------------------------------------------

This week I continued to apply the decorator to functions.
To ensure consistency across the code base, I audited all functions that could benefit from the ``warn_on_args_to_kwargs`` decorator.
To do this, I had to:

1. Identify target functions:

* Identify functions that could benefit from the decorator.
* continue reviewing the code base to identify functions that accept both positional and keyword arguments.

2. Applying the Decorator:

* For each identified function, I added the ``warn_on_args_to_kwargs`` decorator.
* Example:

.. code-block:: python
@warn_on_args_to_kwargs()
def get_actor_from_primitive(
vertices,
triangles,
*,
colors=None,
normals=None,
backface_culling=True,
prim_count=1,
):
3. Updating Unit Tests:

* updated all the unit tests for the functions where the ``warn_on_args_to_kwargs`` decorator is applied to ensure they respect the new format.
* Example:

.. code-block:: python
actr = get_actor_from_primitive(big_verts, big_faces, colors=big_colors)
- You can find more details and the implementation in my pull request: `https://github.com/fury-gl/fury/pull/888 <https://github.com/fury-gl/fury/pull/888>`_.


What Happens Next?
------------------

For week 4, I plan to:

* Continue refining the ``warn_on_args_to_kwargs`` decorator based on feedback from my Peers `Iñigo Tellaetxe Elorriaga <https://github.com/itellaetxe>`_, `Robin Roy <https://github.com/robinroy03>`_, `Kaustav Deka <https://github.com/deka27>`_, my guide: `Serge Koudoro <https://github.com/skoudoro>`_ and the other community members.
* Apply the ``warn_on_args_to_kwargs`` decorator to all the remaining modules and update all the unit tests of these modules too, to respect the desired format.
* Dive deep into the lazy loading functionality based on my research to optimize performance.
* Further engage in code reviews to support my peers and improve our project.

Did I get stuck?
----------------

I didn't get stuck.

Thank you for following my progress. Your feedback is always welcome.
112 changes: 112 additions & 0 deletions docs/source/posts/2024/2024-06-26-week4-wachiou-bouraima.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
WEEK 4: Updating Decorator, Exploring Lazy Loading, and Code Reviews
====================================================================

.. post:: June 26, 2024
:author: Wachiou BOURAIMA
:tags: google
:category: gsoc

Hello everyone,
---------------

Welcome again to my Google summer of code 2024 (GSoC' 2024) journey 2024!.
This week, I focused on updating the ``warn_on_args_to_kwargs`` decorator, applying it across multiple modules, exploring lazy loading, and continuing with code reviews.


Updating the ``warn_on_args_to_kwargs`` decorator
-------------------------------------------------

Based on feedback from my mentor `Serge Koudoro <https://github.com/skoudoro>`_ and peers `Iñigo Tellaetxe Elorriaga <https://github.com/itellaetxe>`_, `Robin Roy <https://github.com/robinroy03>`_, `Kaustav Deka <https://github.com/deka27>`_, I refined the ``warn_on_args_to_kwargs`` decorator and its associated unit tests:

1. Improvements:

- Added conditions to verify if the values of ``from_version``, ``until_version``, and the current version of FURY are respected. This includes handling cases where ``from_version`` is greater than the current version of FURY, ``until_version`` is less than the current version of FURY, and ``until_version`` is greater than or equal to the current version of FURY.
- Ensured the decorator and tests cover a broader range of edge cases.
- Enhanced the warning messages for better clarity and guidance.

2. Doctest Updates:

- Updated the doctest considering the values of `from_version` and `until_version`.
- Moved the doctest from the `def decorator()` function to the root function.

3. Unit Tests:

.. code-block:: python
def test_warn_on_args_to_kwargs():
@warn_on_args_to_kwargs()
def func(a, b, *, c, d=4, e=5):
return a + b + c + d + e
# if FURY_CURRENT_VERSION is less than from_version
fury.__version__ = "0.0.0"
npt.assert_equal(func(1, 2, 3, 4, 5), 15)
npt.assert_equal(func(1, 2, c=3, d=4, e=5), 15)
npt.assert_raises(TypeError, func, 1, 3)
- This ensures robust validation and helps catch potential issues early.


Applying the ``warn_on_args_to_kwargs`` Decorator
-----------------------------------------------

This week, I applied the ``warn_on_args_to_kwargs`` decorator to several modules, ensuring consistent usage and improved code quality. The modules updated include:

- `actors`
- `ui`
- `animation`
- `shares`
- `data`

For each module, I opened a pull request to track the changes and facilitate reviews:

- `actors`: https://github.com/fury-gl/fury/pull/898
- `animation`: https://github.com/fury-gl/fury/pull/899
- `data`: https://github.com/fury-gl/fury/pull/900
- `shares`: https://github.com/fury-gl/fury/pull/901
- `ui`: https://github.com/fury-gl/fury/pull/902


Exploring lazy loading
----------------------

In order to optimize performance, I've started exploring and implementing lazy loading. This week, the focus was on the following points:

- Getting to grips with how the lazy loader works
- Implementing some small script to understand how the lazy loader works
- I also read the SPEC1 document available at `SPEC1 <https://scientific-python.org/specs/spec-0001/>`_
- Understanding the benefits of lazy loading and how it can be applied to the FURY code base
- Planning the integration of lazy loading into the FURY code base

Code sample: `<https://gist.github.com/WassCodeur/98297d7a59b27979d27945760e3ffb10>`_


Peer Code Review
----------------

This week, I continued to dedicate time to reviewing the code of my peers. Specifically, I reviewed Kaustav Deka’s work, providing constructive feedback and suggestions for improvement. You can view the pull request here: `https://github.com/dipy/dipy/pull/3239 <https://github.com/dipy/dipy/pull/3239>`_.


Acknowledgements
----------------

I am deeply grateful to my classmates `Iñigo Tellaetxe Elorriaga <https://github.com/itellaetxe>`_, `Robin Roy <https://github.com/robinroy03>`_, `Kaustav Deka <https://github.com/deka27>`_ for their continuous support and insightful suggestions. Special thanks to my mentor, `Serge Koudoro <https://github.com/skoudoro>`_ , whose expertise and guidance have been invaluable in navigating these technical challenges.


Did I get stuck?
-----------------

Yes, I was a bit confused about understanding lazy loader, but thanks to the help of my mentor `Serge Koudoro <https://github.com/skoudoro>`_ , I was able to understand it better.


What's next?
------------

For the upcoming week, I plan to:

- Implement lazy loading in the FURY code base
- Continue refining the ``warn_on_args_to_kwargs`` decorator based on feedback
- Engage in more code reviews to support my peers
- Prepare to working on the FURY website to improve the documentation and user experience

Thank you for following my progress. Your feedback is always welcome.
74 changes: 74 additions & 0 deletions docs/source/posts/2024/2024-07-01-week-4-robin.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
Week 4: Pipeline Improvements and Taking The Bot Public!
========================================================

.. post:: July 1 2024
:author: Robin Roy
:tags: google
:category: gsoc

Hi, I'm `Robin <https://github.com/robinroy03>`_ and this is my blog about week 4.

My goals for week 4 were to move my Google colab notes to a proper Python script, improve the existing code, and make a working pipeline to upsert data easily. Also, the bot is public now :) Anyone reading this blog could join this `Discord Server <https://discord.gg/NVkE6Qd2bZ>`_ and ask questions right away!

Things I did in Week 4
----------------------

1) **Chunking tutorials and documentation**

Earlier, only files fitting the context window of the embedding model were upserted. This was because otherwise, we'd have to split the file in half and lose the overall context. This will lead to information loss and retrieval will be messy. Now, I decided I'd upsert everything by splitting information properly. By "properly", what I mean is it won't be a random split, and there'll be logical reasoning behind every chunk.

This area is still actively studied, and the whole concept is to find ideal chunks which are self-sufficient and contain the most information. This `notebook <https://github.com/FullStackRetrieval-com/RetrievalTutorials/blob/main/tutorials/LevelsOfTextSplitting/5_Levels_Of_Text_Splitting.ipynb>`_ details 6 different approaches, I read through them and some of their associated literature and decided we'll use `Recursive Character Text Splitting` and `Document Specific Splitting` for now. There is no major reason for this, I just felt it'll work well for now (a reasoning-backed approach will come in a few weeks). There is a lot of experimentation we could do here, a better chunking will result in better ``references`` generation and so on.

So this is our current process
- if normal function/class definition: no splitting, chunk as it is.
- if rst files, use the ``rst parser`` and split them with a chunk size of ~8000 tokens (max llama could take). RST files in FURY contain documentation & blog posts.
- if tutorial, try chunking as it is, if not possible split at 8000 tokens.

Function/class definitions are generally under 8000 so I've not done explicit checks for now, the model will trim the remaining if longer (I found some long classes later).

2) **Move colab files to a proper Python script**

I did all the upsertion and experiments on colab. It is messy and can't be used in production. We need a one-click approach to upsertion. Something like point to `fury` directory and it should do everything. So I took the messy colab code and made a python script from it.

One of my key goals is to separate core application logic from LLMs/Database providers. We should be able to swap them as needed without much fuss. I'll talk more about this in week 5.

3) **Taking the bot public!**

The whole point of making the bot is to help improve the productivity of FURY developers. So I decided to take it public on `this discord server <https://discord.gg/NVkE6Qd2bZ>`_. You could use it today! (actually, you could've used it from the 20th of last month, this blog got delayed😢)

I'll observe what people are asking and then iterate towards making the bot better in that area. I think it'll be better than making the bot good on what I believe is the best.

4) **Minor bugfixes and stuff**

Did some minor bug fixes on things like the Discord bot generation cutoff and error handling improvements. It was Discord message limit (<=2000) that caused the generation to cut off, I split the message into parts to fix that. Error handling was improved generally everywhere. I'll need to bring logging later.


Minor Sidequest
~~~~~~~~~~~~~~~

This is in no way related to FURY, but it was fun so I thought I'd add it here :)

So after midterms, I decided to go back home, to maximally use my time I searched for things to do and found a local FOSS event (`link <https://x.com/FOSSUnitedKochi/status/1804763181274759645>`_). It was done by FOSS United Kochi and it's one of the major FOSS events in my state (Kerala, India). Met some Pythonistas! Explained what FURY is to them. I also ended up finding some lore (`link <https://www.gnu.org/education/edu-system-india.html>`_) about how GNU/Linux spread in Kerala, India. Also found some old FOSS event pictures (`this <https://www.flickr.com/photos/pce/245170427/in/photostream/>`_ one is talking about Python, 2003 World of Python). This was my first FOSS event outside campus so it was fun :)


What is coming up next week?
----------------------------

- Benchmarking
- Architecture Update

Did you get stuck anywhere?
---------------------------

No, I did not get stuck. This week was more of learning and experimentation so I think it's normal what I encountered.

LINKS:

- `Discord Server <https://discord.gg/NVkE6Qd2bZ>`_
- `A Text Splitting Guide <https://github.com/FullStackRetrieval-com/RetrievalTutorials/blob/main/tutorials/LevelsOfTextSplitting/5_Levels_Of_Text_Splitting.ipynb>`_
- `GNU Case of Kerala <https://www.gnu.org/education/edu-system-india.html>`_
- `2003 World of Python <https://www.flickr.com/photos/pce/245170427/in/photostream/>`_
- `FOSS United Kochi <https://x.com/FOSSUnitedKochi/status/1804763181274759645>`_
- `Robin :) <https://github.com/robinroy03>`_

Thank you for reading!
Loading

0 comments on commit f873d5d

Please sign in to comment.