Skip to content

JoannaCai/llm.go

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

18 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

llm.go

Status GitHub Issues GitHub Pull Requests License

GPT-2 implementation written in go only using the standard library.

🪞 Quick start

Install python dependencies, output tokenized dataset

make setup

Run the training script:

make train

This will run go run ./cmd/traingpt2/main.go

Run the testing script:

make test

This will run go run ./cmd/testgpt2/main.go

TODO

  • Tokenize input text, the implementation of this is incorrect. Need to do pair matching not tries
  • Very slow, need to improve performance.
  • It runs in WASM but using WebGPU bindings might be fun.
  • More refactoring.
  • Running as CLI.

🖋️ License

See LICENSE for more details.

🎉 Acknowledgements

  • This is a fork of Andrej Karpathy's llm.c written in pure go.

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Go 66.1%
  • Python 20.1%
  • JavaScript 12.7%
  • Other 1.1%