model2Vec Files

What's Changed

• feat: disable progress bars, add hf hub as a direct dependency by @stephantul in https://github.com/MinishLab/model2vec/pull/345
• feat: Replace pca by @stephantul in https://github.com/MinishLab/model2vec/pull/344
• feat: export classifier and regressor pipelines to ONNX by @serhiizghama in https://github.com/MinishLab/model2vec/pull/346
• feat: make onnx export a function by @stephantul in https://github.com/MinishLab/model2vec/pull/347
• feat: add onnx model cards by @stephantul in https://github.com/MinishLab/model2vec/pull/349
• fix: add distillation duplicate guard by @stephantul in https://github.com/MinishLab/model2vec/pull/350
• fix: start of word tokens for added by @stephantul in https://github.com/MinishLab/model2vec/pull/351
• fix: trainer uses best checkpoint by @stephantul in https://github.com/MinishLab/model2vec/pull/352
• fix: onnx export correctly ignores unk by @stephantul in https://github.com/MinishLab/model2vec/pull/353
• fix: unk token for unigrams by @stephantul in https://github.com/MinishLab/model2vec/pull/354
• feat: new integration test by @stephantul in https://github.com/MinishLab/model2vec/pull/355
• feat: make setters for max_length by @stephantul in https://github.com/MinishLab/model2vec/pull/356
• chore: typing on loading should return a T, not a StaticModel by @stephantul in https://github.com/MinishLab/model2vec/pull/357
• fix(train): pad training batches with the model's pad id by @serhiizghama in https://github.com/MinishLab/model2vec/pull/358
• fix(train): drop unknown tokens during training and inference by @serhiizghama in https://github.com/MinishLab/model2vec/pull/359
• docs: Update main readme by @Pringled in https://github.com/MinishLab/model2vec/pull/361
• fix(inference): pass the token through when loading the encoder by @serhiizghama in https://github.com/MinishLab/model2vec/pull/362
• feat: max length behavior by @stephantul in https://github.com/MinishLab/model2vec/pull/363
• fix(onnx): export vocabulary-quantized models with their token mapping and weights by @serhiizghama in https://github.com/MinishLab/model2vec/pull/364
• feat: small helper method to reduce overlap by @stephantul in https://github.com/MinishLab/model2vec/pull/365
• docs: fix classifier docstring by @stephantul in https://github.com/MinishLab/model2vec/pull/366
• feat: higher speed encoding by @stephantul in https://github.com/MinishLab/model2vec/pull/367
• fix: norm underflow for float16 by @stephantul in https://github.com/MinishLab/model2vec/pull/369
• feat: add allow list for paths by @stephantul in https://github.com/MinishLab/model2vec/pull/370
• feat: remove loggit mapping from training by @stephantul in https://github.com/MinishLab/model2vec/pull/371
• feat: add token dropout to training pipeline by @stephantul in https://github.com/MinishLab/model2vec/pull/373
• feat: add pairwise trainer by @stephantul in https://github.com/MinishLab/model2vec/pull/372
• fix(train): keep the trainer's max_length independent of the static model's tokenizer truncation by @serhiizghama in https://github.com/MinishLab/model2vec/pull/374
• fix: disable padding during distill and inference by @stephantul in https://github.com/MinishLab/model2vec/pull/376
• docs: Add Greptile acknowledgement by @Pringled in https://github.com/MinishLab/model2vec/pull/377
• fix: divide by length 1 instead of 1e-16 by @stephantul in https://github.com/MinishLab/model2vec/pull/380
• fix: stop the trainer's tokenizer from padding by @stephantul in https://github.com/MinishLab/model2vec/pull/379
• feat: pair and similarity head with 0 layers have no layers by @stephantul in https://github.com/MinishLab/model2vec/pull/383
• feat: train pair models with InfoNCE by @stephantul in https://github.com/MinishLab/model2vec/pull/381
• feat: add other metrics to loop progress bar by @stephantul in https://github.com/MinishLab/model2vec/pull/384
• feat: add contribution guidelines by @stephantul in https://github.com/MinishLab/model2vec/pull/386
• feat: add support for datasets in training by @stephantul in https://github.com/MinishLab/model2vec/pull/387
• feat: add explicit kwargs to all trainable by @stephantul in https://github.com/MinishLab/model2vec/pull/388
• feat: make max length argument explicit by @stephantul in https://github.com/MinishLab/model2vec/pull/389
• feat: speed up training, remove pad token requirement by @stephantul in https://github.com/MinishLab/model2vec/pull/390
• feat: add focal loss by @stephantul in https://github.com/MinishLab/model2vec/pull/391
• fix: download readme and respect subfolder by @stephantul in https://github.com/MinishLab/model2vec/pull/392
• fix: add token dropout to pair trainer by @stephantul in https://github.com/MinishLab/model2vec/pull/393
• remove tutorials by @stephantul in https://github.com/MinishLab/model2vec/pull/394
• fix: random seed controls splitting by @stephantul in https://github.com/MinishLab/model2vec/pull/396
• docs: update docs for training by @stephantul in https://github.com/MinishLab/model2vec/pull/397
• chore: update version -> v0.10.0 by @stephantul in https://github.com/MinishLab/model2vec/pull/398

New Contributors

• @serhiizghama made their first contribution in https://github.com/MinishLab/model2vec/pull/346

Full Changelog: https://github.com/MinishLab/model2vec/compare/v0.9.0...v0.10.0