Skip to content

Commit

Permalink
Add createByModelName example
Browse files Browse the repository at this point in the history
  • Loading branch information
danny50610 committed Aug 20, 2023
1 parent ae4218d commit a162176
Show file tree
Hide file tree
Showing 2 changed files with 10 additions and 2 deletions.
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ $RECYCLE.BIN/

# End of https://www.toptal.com/developers/gitignore/api/windows,linux,macos,composer

tinker.php
_tinker*.php
.phpunit.result.cache
.phpunit.cache/
composer.lock
10 changes: 9 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,19 @@ composer require danny50610/bpe-tokeniser

### GPT-4 / GPT-3.5-Turbo (cl100k_base)
```php
<?php
use Danny50610\BpeTokeniser\EncodingFactory;

$enc = EncodingFactory::createByEncodingName('cl100k_base');

var_dump($enc->decode($enc->encode("hello world")));
// output: string(11) "hello world"
```

```php
use Danny50610\BpeTokeniser\EncodingFactory;

$enc = EncodingFactory::createByModelName('gpt-3.5-turbo');

var_dump($enc->decode($enc->encode("hello world")));
// output: string(11) "hello world"
```

0 comments on commit a162176

Please sign in to comment.