Home#
This course focuses on advanced deep learning techniques for natural language processing using the NVIDIA NeMo framework and DGX H100 server. It emphasizes the training, optimization, and deployment of Large Language Models (LLMs), including practical application development. Students will gain hands-on LLM development experience through team projects.
Table of Contents#
Learning Objectives#
Master the use of NVIDIA NeMo framework and DGX H100 server.
Understand and practice LLM architecture and training methods.
Acquire techniques for LLM training and optimization using custom datasets.
Learn LLM development methods in distributed learning environments.
Develop the ability to implement RAG (Retrieval-Augmented Generation) systems.
Cultivate skills to design and develop real-world LLM-based applications.
Evaluation#
Attendance and Participation (10%)
Evaluation method: Weekly attendance check and in-class participation
Evaluation timing: Every week
Weekly Practical Assignments (20%)
Evaluation method: Submission of weekly practice results
Evaluation timing: Weeks 2-7, 9-14
Midterm Project Presentation (20%)
Evaluation method: Team project interim results and presentation
Evaluation timing: Week 8
Final Project (50%)
Evaluation method: LLM application development results, technical documentation, presentation, peer evaluation
Evaluation timing: Week 15
Course Materials#
Lecture Note: https://deepnlp2024.halla.ai
Main textbook: NVIDIA NeMo official documentation and tutorials
Additional references: Latest papers on LLMs, NVIDIA technical blog posts, GitHub repositories
Prerequisites#
Python programming (Intermediate level or above)
Basics of machine learning and deep learning
Introduction to natural language processing
Additional Notes#
DGX H100 server account required, to be issued before the course
Team projects will be conducted in groups of 3-4 members
Course content may be partially modified to reflect the latest technology trends
Weekly Practice Assignment Guidelines#
Weekly assignments due by Friday midnight
Code management and submission through GitHub recommended
Project Guidelines#
Midterm project: Fine-tuning and performance improvement of LLM for specific tasks
Final project: Developing LLM-based applications solving real-world problems
Project deliverables should include code, technical documentation, and presentation materials
Open-source contribution or paper writing can be substituted (prior consultation required)
Changelog#
See the CHANGELOG for more information.
Contributing#
Contributions are welcome! Please see the contributing guidelines for more information.
License#
This project is released under the CC-BY-4.0 License.