transformer

Transformer is a TensorFlow implementation of the architecture introduced in the Attention Is All You Need paper. It was created as a readable and relatively modular reference for understanding and experimenting with Transformer-based machine translation. The updated implementation corrects issues involving masking, positional encoding, and other parts of the original code. It adds components such as byte-pair encoding and shared weight matrices. Training and evaluation are demonstrated with the IWSLT 2016 German-to-English translation dataset. The repository includes preprocessing, training, inference, evaluation, configurable hyperparameters, pretrained checkpoints, and BLEU-based translation results.