Many people believe open source is shutting the door on AI written code. I read two studies from this year that checked this properly and tried to validate with understanding from open source projects I am involved in.
During training pipeline issues can manifest themselves as data deficiency from the outside. Here are few things you can do
During training a data mixture config will say something like code (data domain) is 10 percent of the run. That number is a plan and it is not what the model actually experiences during training.
When a large training run is slow, the usual dashboard shows GPU utilization, memory used and NVLink traffic per GPU. None of these tells a researcher which parallelism dimension is costing time.
This is part two of two posts on how a model can work with an input that is much longer than its context window. In part one I explained the recursive language model (RLM).
This is part one of two posts on how a model can work with an input that is much longer than its context window and also improve how its reasons over this long context. Language models have a fixed context window.
Meta-Harness This is a continuation of my thread on agentic system optimization. In this one I want to show how Meta-Harness works in practice.
Continuing my thread on agentic systems optimization. Harness optimization is all the rage so lets look at one technique.
Continuing my thread on agentic systems optimization I found AgentFactory concept interesting. It is a recent paper to optimize the model and the agent workflow together, rather than treating the underlying model as fixed and only searching over prompts or workflow structure.
Yesterday I wrote about GEPA which is a prompt optimizer that uses an LLM to read execution traces and evaluator feedback and rewrite one prompt at a time in a agentic system. By default, GEPA rotates through prompts , while choosing the candidate to mutate using an instance-level Pareto frontier over validation examples.
GEPA is a prompt optimizer for agentic systems that have more than one step. Here is how it works, with a simple example.
Suppose a student knows three valid ways to solve a problem. They can use algebra, draw a diagram, or reason through the numbers directly.
AutoJev-27B ranks second on Hugging face on Jev benchmarks. Here is a quick summary of architecture -
Iceberg 1.12 introduces server-side scan planning work in Iceberg REST Catalog. I have mixed feeling not specifically about this change but moving more work towards a central control plans from engines.
A multimodal model has two main parts: encoders that turn images, video, or audio into embeddings, and the LLM backbone, which is the language model that takes those embeddings together with the text tokens and does most of the computation. Memory profile a. The encoder for images, video, or audio is usually small, but long samples create a lot of activation memory.
It was interesting to compare MoE Parallel Folding in Megatron with JAX/TPU architecture. MoE Parallel Folding lets attention layers and MoE layers use different parallelism settings in the same model.
NVIDIA cuObject lets applications read and write S3 compatible object storage directly into GPU memory over RDMA. The concept has strong efficiency promise.
Trino can now read Iceberg tables with encrypted Parquet files. Here is how encrypted Parquet support has evolved across Spark, Iceberg and Trino.
RL post-training for agents (2 of 2) 1. Rollouts and training a. Training and Inference infra can be colocated and same GPUs alternate between generating and training, and weights are resharded between the training layout and the inference layout. b. Colocated GPUs wait on the longest trajectory.
RL post-training for agents (1 of 2) What it takes to post-train a model to operate reliably as an agent. This is a 2 post series.
