Explore cutting-edge research in language models and discover our latest breakthroughs in large model training, optimization, and applications. Our research advances the boundaries of artificial intelligence technology.
We developed a new-generation training method for multimodal large language models. While ensuring outstanding performance in cross-modal understanding and generation tasks, our training method consumes fewer resources and requires shorter training times. This lays the foundation for training more general-purpose AI systems.
We propose a new parameter-efficient fine-tuning method that preserves pre-trained knowledge while adapting to new tasks by updating only a small number of parameters. This method achieves performance comparable to full-parameter fine-tuning on multiple benchmarks.
We developed a model architecture capable of processing ultra-long contexts (128K tokens), addressing the limitations of existing models in long-document understanding. This model excels in long-document summarization and question-answering tasks.
We propose a new alignment technique that significantly reduces the probability of models generating harmful content through multi-stage training and reinforcement learning constraints, while maintaining model creativity and utility.
We built a large-scale pretrained model supporting 50+ languages. Through an innovative language representation sharing mechanism, we significantly improved performance for low-resource languages while maintaining quality for high-resource languages.
We developed a new inference optimization framework that improves the inference speed of large language models by over 3x through dynamic computation allocation and caching strategies, while maintaining output quality.
We propose a method to update model knowledge without retraining, allowing precise modification of specific knowledge in the model without affecting other capabilities, addressing a key challenge in large model knowledge updates.