Posts

Put simply, hallucination is a language model’s behavior that creates plausible but unsupported statements. It has been a known weakness since the earliest language models and remains unsolved. Many approaches—such as Retrieval-Augmented Generation (RAG) and reinforcement learning (RL) —reduce the rate but do not eliminate it. So, why do language models hallucinate? First, some questions cannot be answered with certainty from the information at hand. With limited context, there is no way to guarantee a correct answer.

Specifications are documents that describe how software should behave and what functions it must perform. In other words, they are a separate form of implementation that exists independently of the actual program. By leveraging specifications, we can verify that our software is constructed correctly, which improves its safety and reliability. Yet maintaining specifications is hard and boring. Because specifications do not directly improve performance or features, most companies neglect them. Once a specification falls behind, it quickly degrades into legacy that nobody reads or maintains.

The history of architecture can be divided into before and after the development of reinforced concrete. Steel rebar and concrete each have strength against different types of forces: tension and compression. When combined, both materials’ advantages can be fully utilized. Moreover, the two materials have nearly identical thermal expansion coefficients, preventing cracks from forming even with temperature changes. There are also attempts to develop an equivalent of reinforced concrete in the AI field: Neuro-Symbolic AI.