> For the complete documentation index, see [llms.txt](https://th-koeln-1.gitbook.io/th-koeln-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://th-koeln-1.gitbook.io/th-koeln-docs/a-glossary-of-language-oriented-ai-work-in-progress.md).

# A glossary of language-oriented AI (work in progress)

<figure><img src="https://github.com/ITMK/AI_Literacy/blob/main/images/GenAI_ITMK.jpg?raw=true" alt=""><figcaption></figcaption></figure>

[The curriculum](https://github.com/ITMK/AI_Literacy/)

Author: [Ralph Krüger](https://www.th-koeln.de/en/person/ralph.krueger/)

## A

### Activation function

### Alignment

### Array

### Artificial general intelligence (AGI)

### Artificial intelligence (AI)

### artificial narrow intelligence (ANI)

### Attention

### Attention mask

### Attention score

### Attention weight

### Auto-regressive decoder

## B

### Backpropagation

### Backward pass

### BART (Bidirectional and Auto-Regressive Transformer)

### Beam search

### BERT (Bidirectional Encoder Representations from Transformers)

### BertViz

### Bias value

### Bidirectional encoder

### Byte-pair encoding (BPE)

## C

### Causal language modelling

### Chain of thought (CoT)

### Chain rule

### Character-based tokenisation

### Classification

### Concatenation

### Contextualised embedding

### Context window

### Continuous Bag of Words (CBOW)

### Convolutional neural network (CNN)

### Cosine similarity

### Cross-entropy loss

## D

### Decoder

### Decoder-only language model

### Decontextualised embedding

### Deep learning (DL)

### Deep neural network

### Derivative

### Dot product

## E

### Embedding

### Embedding matrix

### Encoder

### Encoder-decoder language model

### Encoder-only language model

### Euclidean distance

## F

### Feedforward neural network

### Few-shot

### Fine-tuning

### Forward pass

## G

### General-purpose artificial intelligence (GPAI)

### GPT (Generative Pre-Trained Transformer)

### Gradient

### Gradient descent

### Greedy decoding

## H

### Hidden layer

### Hugging Face

### Hyperparameter

## I

### In-context learning

### Inference

### Input embedding

### Input layer

## J

## K

### Key matrix

### Key vector

### Knowledge distillation

### Knowledge-enhanced language model

### Knowledge graph

## L

### Layer normalisation

### Linear layer

### LLM-as-a-judge

### Logits vector

### Loss

### Loss function

### Low-rank adaptation (LoRA)

## M

### Machine learning (ML)

### Masked language modelling

### Masked token prediction

### Massively multi-task learning

### Matrix

### Matrix product

### Modality

### Modality encoder

### Modality interface

### Model compression

### Multi-head attention

### Multimodal language model

## N

### Natural language generation (NLG)

### Natural language processing (NLP)

### Natural language understanding (NLU)

### Negative log-likelihood

### Neural network

### Network error

### Neuron

### Next token prediction

### Non-linearity

### NumPy

## O

### One-hot vector

### Optimisation

### Output layer

## P

### Parameter-efficient fine-tuning (PEFT)

### Positional encoding

### Preference tuning

### Pre-training

### Prompt engineering

### Prompting

### Pruning

### PyTorch

## Q

### Quantisation

### Quantized low-rank adaptation (QLoRA)

### Query matrix

### Query vector

## R

### ReAct (Reason and Act)

### Rectified Linear Unit (ReLU)

### Recurrent neural network (RNN)

### Representation learning (RL)

### Regression

### Reinforcement learning

### Residual connection

### Retrieval-augmented generation (RAG)

## S

### Scalar

### Scaling law

### Scaled dot-product attention

### Self-attention

### Self-supervised learning

### Sentence embedding

### Shallow neural network

### Skip-Gram

### Softmax

### Subword

### Subword-based tokenisation

### Supervised learning

## T

### Tensor

### TensorFlow

### Test-time scaling

### Text embedding

### Tiktokenizer

### Token

### Tokenisation

### Train-time scaling

### Transformer

### Transposition

## U

### Unigram

### Unimodal language model

### Unsupervised learning

## V

### Value matrix

### Value vector

### Vector

## W

### Weight

### Weighted sum

### Weights matrix

### Word-based tokenisation

### Word embedding

### WordPiece

## X

## Y

## Z

### Zero-shot<br>


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://th-koeln-1.gitbook.io/th-koeln-docs/a-glossary-of-language-oriented-ai-work-in-progress.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.