理工学研究科Graduate School of Science and Engineering
HUI500X3（人間情報学 / Human informatics 500）大規模言語モデルを用いた生成型AIGenerative AI with Large Language Models
GUO AOAo GUO
授業コードなどClass code etc
| 学部・研究科Faculty/Graduate school | 理工学研究科Graduate School of Science and Engineering |
| 添付ファイル名Attached documents | |
| 年度Year | 2024 |
| 授業コードClass code | YB040 |
| 旧授業コードPrevious Class code | |
| 旧科目名Previous Class title | |
| 開講時期Term | 秋学期授業/Fall |
| 曜日・時限Day/Period | 月1/Mon.1 |
| 科目種別Class Type | |
| キャンパスCampus | 小金井 / Koganei |
| 教室名称Classroom name | 小西館‐W302 |
| 配当年次Grade | |
| 単位数Credit(s) | 2 |
| 備考（履修条件等）Notes | |
| 実務経験のある教員による授業科目Class taught by instructors with practical experience | |
| カテゴリーCategory | 応用情報工学専攻 |
すべて開くShow all
すべて閉じるHide All
Outline (in English)
This course is designed to help students have a deep understanding of generative AI, with a focus on large language models (LLMs). It covers the fundamentals of LLMs, practical skills for their implementation, and the utilization of LLMs in interdisciplinary research.
授業で使用する言語Default language used in class
英語 / English
授業の概要と目的（何を学ぶか）Outline and objectives
This course is designed to help students have a deep understanding of generative AI, with a focus on large language models (LLMs). It covers the fundamentals of LLMs, practical skills for their implementation, and the utilization of LLMs in interdisciplinary research.
到達目標Goal
In this course, students will gain necessary knowledge of generative AI with large language models (LLMs), including their theory, development, and important research topics. Students will be able to implement LLMs using several fine-tuning techniques, develop LLM-based systems with various common purposes, and evaluate their performance. By the end of this course, students are expected to be proficient in applying LLMs to their own research projects.
授業で使用する言語Default language used in class
英語 / English
授業の進め方と方法Method(s)（学期の途中で変更になる場合には、別途提示します。 /If the Method(s) is changed, we will announce the details of any changes. ）
This course is conducted mainly in two parts. Firstly, basic knowledge of large language models (LLMs) will be introduced. As part of this learning process, students will be asked to provide feedback and submit reaction papers on this basic knowledge. Then, the construction of LLMs for various common purposes will be practiced, along with efficient techniques for applying LLMs in scientific research. Students will be involved in developing these LLMs to gain a deeper understanding. Finally, students will be asked to present a project of their interest using LLMs as their final presentation.
アクティブラーニング（グループディスカッション、ディベート等）の実施Active learning in class (Group discussion, Debate.etc.)
あり / Yes
フィールドワーク（学外での実習等）の実施Fieldwork in class
なし / No
授業計画Schedule
授業形態/methods of teaching：対面/face to face
※各回の授業形態は予定です。教員の指示に従ってください。
1[対面/face to face]：Course Introduction
Course overview, basic concepts of generative AI and Large Language Models (LLMs), development of LLMs and environment setup.
2[オンライン/online]：Review of Neural Networks and Deep Learning
Review of neural networks architecture, activation functions, backpropagation, optimization, etc.
3[対面/face to face]：Fundamentals of Natural Language Processing
Introduction to tokenization, word embedding, and language modeling.
4[オンライン/online]：Transformers
Introduction to transformer architecture, attention mechanism, and encoder-decoder.
5[対面/face to face]：Large Language Models (I)
Introduction to basic LLMs: GPT, BERT, RoBERTa, T5, and Multilingual BERT
6[オンライン/online]：Large Language Models (II)
- Prompt-based learning regarding zero-shot, one-shot, and few-shot learning
- Chain-of-thought - RLHF
7[対面/face to face]：Fine-Tuning of Large Language Models
- Benchmark for LLMs
- Implementation of LLM for emotion analysis
- Setup for dialogue system
8[オンライン/online]：Named Entity Recognition (NER)
Introduction to dataset, preprocessing, evaluation metrics, and Implementation of LLM for NER.
9[対面/face to face]：Summary Generation and Question Answering System
Introduction to summary generation and question answering system, regarding dataset, evaluation metrics and their implementation.
10[オンライン/online]：Task-oriented and Open-domain Dialogue System
Introduction to their development, state-of-the-art models, limitations, and their implementation.
11[対面/face to face]：Image Generation with Large Language Model
Introduction to some popular models for image generation. Video-to-text translation will be also introduced.
12[オンライン/online]：Advanced LLMs and Efficient Fine-tuning
Introduction to advanced LLMs (e.g., LLaMA and OpenLM), and efficient fine-tuning method (e.g., LoRA, QLoRA, Adapter Tuning, Prompt Tuning, etc.)
13[オンライン/online]：Crowdsourcing for LLMs
Introduction to crowdsourcing platform and setup for Amazon Mechanical Turk.
14[対面/face to face]：Final Presentation
Students present their projects.
授業時間外の学習（準備学習・復習・宿題等）Work to be done outside of class (preparation, etc.)
Homework and final presentation/project.
It takes four hours for weekly pre-study and assignments on average.
テキスト（教科書）Textbooks
Handouts and prints will be distributed.
参考書References
1. Understanding Large Language Models: Learning Their Underlying Concepts and Technologies
2. Generative Deep Learning: Teaching Machines to Paint, Write, Compose, and Play (Second Edition)
3. 大規模言語モデル入門 (JAPANESE EDITION)
4. Pythonでつくる対話システム (JAPANESE EDITION)
成績評価の方法と基準Grading criteria
Assignment (both in class/homework) 50% + Final presentation/project 50%
学生の意見等からの気づきChanges following student comments
本年度新規科目につきアンケートを実施していません
学生が準備すべき機器他Equipment student needs to prepare
1. Basic knowledge of Machine Learning
2. Basic knowledge of Python, and some packages such as Numpy, Pandas, Matplotlib, PyTorch, etc.