Transformer Day: Get miniGPT up and running

[ramon-astudillo]

Objective: Integrate miniGPT https://github.com/karpathy/minGPT into the toolkit.

• Add a basic notebook labs/notebooks/transformers.pynb that runs a miniGPT.from_pretrained decoding with an example
  • Port only the minGPT code needed (e.g. under labs/lxmls/miniGPT).
    • Less code is better for clarity. Code is already pretty compact so it may be ok as is.
    • Add License and ensure we attribute properly
  • Load a basic model (GPT-2 large? XL?)
  • Check a basic text completion exercise works, where we can give different prefixes are completions make more or less sense
  • Other proposals @venelink @gonmelo ?

Branch: https://github.com/LxMLS/lxmls-toolkit/tree/transformer-day

Expected Finishing date: Ideally before June 12 meeting. If not during that week.

More ambitious objectives would include

• Add a fine-tuning example