Neural Machine Translation for English & Uzbek
Experience bidirectional alignment, resolve English (SVO) to Uzbek (SOV) structural word-reordering, and observe token-level attention distribution in real-time.
Translation will appear here...
Token Alignment Flow
Visualizing the cross-lingual word order shift from English (SVO) to Uzbek (SOV agglutinative).
2D Bahdanau Attention Matrix ($\alpha_{ij}$)
Each cell represents the softmax attention probability assigned by decoder step $t$ to encoder hidden state $i$.
Architectural Formulation & Linguistic Bridge
How Bahdanau Additive Attention solves the information bottleneck for agglutinative languages.
Additive Attention Scoring
Unlike dot-product attention, Bahdanau uses a learnable feedforward network to calculate compatibility between previous decoder state $s_{t-1}$ and all encoder states $h_i$:
score(s_{t-1}, h_i) = v_a^\top \tanh\left(W_a s_{t-1} + U_a h_i\right)
Where $W_a \in \mathbb{R}^{d \times d}$, $U_a \in \mathbb{R}^{d \times 2d}$ (for bidirectional encoder), and $v_a \in \mathbb{R}^{d}$.
Softmax Probability Distribution
The raw alignment energy scores are normalized across source sequence tokens using the softmax function:
\alpha_{t, i} = \frac{\exp(\text{score}(s_{t-1}, h_i))}{\sum_{k=1}^{T_x} \exp(\text{score}(s_{t-1}, h_k))}
$\sum_{i=1}^{T_x} \alpha_{t, i} = 1.0$. This probability distribution forms the rows of our 2D heatmap matrix!
Dynamic Context Vector
Instead of forcing the entire English sentence into a single static hidden vector, decoder step $t$ computes a tailored context vector:
c_t = \sum_{i=1}^{T_x} \alpha_{t, i} h_i
The Uzbek word is predicted conditioned on $[y_{t-1}; c_t]$, giving direct access to relevant source words at every decoding step.
Linguistic Structure: SVO vs SOV
English follows Subject-Verb-Object (e.g., "I [S] love [V] books [O]"), whereas Uzbek strictly adheres to Subject-Object-Verb ("Men [S] kitoblarni [O] yaxshi ko'raman [V]").
Notice how in the attention matrix, the Uzbek verb attends back to the early English verb, forming a characteristic non-diagonal alignment pattern!