From Memory to Self-Evolution: A Critical Step in Agent's Journey Towards RSI

Recently, top AI laboratories around the world have been releasing a strong signal (indicating a significant development or trend):AI is moving from helping humans complete tasks to participating in its own research and development as well as improvement.

In September, OpenAI announced that it had achieved the initial goals of its “automated AI research intern” program and plans to create true “automated AI researchers” by March 2028. Jakub Pachocki, OpenAI’s Chief Scientist, also published a lengthy article discussing the potential of Recursive Self-Improvement (RSI). On September 21, OpenAI further identified the development of automated AI researchers and the continuous improvement of AI research and development capabilities as key strategic priorities for the next phase.

OpenAI released an official strategic statement on September 21, 2026, outlining automation of AI researchers and recursive self-improvement as key goals for the next phase of its development.

On the other hand, Anthropic also released a set of noteworthy data: as of August this year, Claude has already... 26% of the internal AI research and development work is led by our team.More and more research and development tasks that were originally carried out by researchers are now being executed by AI under human supervision.

From OpenAI to Anthropic, RSI is moving from a distant technological concept to a core part of the research agendas at top global laboratories. AI is no longer just a tool for solving external problems; it is also beginning to play a role in driving the development of its own capabilities.

However, the most critical aspect of this ongoing evolution is:

How can AI turn past explorations and trial and errors into the foundation for the next step of progress, rather than starting from scratch every time a new task arises?

Memory is the key foundation that enables agents to continuously improve themselves. It ensures that each exploration leaves behind not only the results of the task but also experience that can be inherited and reused by subsequent tasks, providing opportunities for the Agent’s capabilities to evolve beyond a single task and to continuously improve through ongoing accumulation.

01 | From Experience Accumulation to Continuous Evolution:

How Memory Supports RSI

The key to enabling the Agent to continuously improve its capabilities during execution is to allow experience to transcend a single task:Effective methods are derived from past explorations, accurately applied in new tasks, and continuously refined and optimized based on the results of those tasks. In this process, memory writing and retrieval play two crucial roles that are closely connected.

Memory writing requires identifying effective methods, failure conclusions, and their applicable conditions from complex execution trajectories, so that past tasks can truly leave behind reusable experience; memory retrieval, on the other hand, needs to accurately find applicable historical methods and key evidence based on the requirements of new tasks, to avoid missing out on experience and making incorrect transfers.

Furthermore, task feedback can also be used to verify the actual value of that experience. By continuously updating the memory content and using effective feedback to train the memory-based models, the memory's own capabilities can be enhanced, creating a mutually reinforcing process between experience accumulation and Agent improvement.

02 | Learning and Evolution of Memory:

Let the Agent continuously improve from its experience.

Based on the needs for experience accumulation, cross-task reuse, and continuous improvement in Agent RSI, MemoraX has been built with the following considerations in mind:Experience-based learning, experience transfer, and feedback optimizationA memory capability system centered around the core principle of efficient information retention, which involves the construction of high-quality memory data, dynamic organization of memory content, and optimization based on real-world task feedback. This system continuously trains the underlying memory models, enabling the Agent to learn from historical experiences and continuously improve its decision-making and execution processes.

Learn from tasks: Turn the accumulation of experience into an optimizable model capability

During the execution of the Agent's tasks, continuous verification of results and adjustment of strategies will generate new experiences.Successful decisions provide reusable methods for subsequent tasks, while verified conclusions about failures help the Agent eliminate ineffective solutions and reduce the need for repeated trial and error.However, the value of experience often becomes apparent only in subsequent tasks. Accurately summarizing the past does not necessarily mean that it can effectively support the future. Therefore, the process of learning from experience requires not only identifying the experiences that are worth retaining but also preserving the evidence that supports them, as well as the environmental constraints and conditions under which they were applied. This is to prevent experiences from being overlooked or misused, which is far beyond what simple information summarization and compression can achieve.

In response to this challenge, MemoraX Write memories into a learnable strategy that is modeled for future task benefits, and continuously build a complete system that covers environment construction, data collection, sample creation, and large-scale Agentic RL training.In an interactive and verifiable task environment, execution trajectories, memory usage records, and validation feedback are collected to construct training samples for decision-making and subsequent usage effects. Reward signals are designed based on the quality of completion, execution efficiency, and the number of trial-and-error attempts in subsequent tasks, in order to optimize the memory writing strategy. This process extends the optimization goal of the model from the accuracy of information extraction to other aspects as well.The actual impact of memory on the quality of subsequent tasks, execution efficiency, and the ability to repeat experiments and make correctionsLearning is more beneficial for developing memory strategies that will be useful for future tasks.

Migrate from experience: Let past lessons guide new tasks

Experience can be preserved, but that does not necessarily mean it can be effectively applied in new tasks. When facing different task objectives, environmental constraints, and execution phases, the Agent needs to accurately identify the historical methods that are applicable, and avoid the interference of irrelevant information as well as the misapplication of past experience.

For this reason, MemoraX willDecoupling memory content from access structureDynamically adjust the memory access points and associated relationships based on the current task, so that the same historical information can meet different usage requirements.

At the retrieval level, let's go further...Learn the matching relationship between the current task requirements and historical key evidence through model training.Combining fine-grained evidence recognition with similar memory disambiguation to enhance the ability to locate critical experiences, distinguishing between 'topic-relevant' and 'evidence-accurate' information.

During the hands-on usage phase, MemoraX takes into account the current task status and the progress of execution.Transform the general experience accumulated over time into action guidelines that have clear objectives, operational constraints, and validation criteria.And dynamically adjust according to the task phase, allowing historical experience to truly contribute to the Agent's decision-making and execution.

Evolving from feedback: Continuously optimizing memory performance through execution results

In the continuous iteration of RSI (Relative Strength Index), historical experience is not always effective. New tasks may either validate existing methods or reveal their limitations.How to use feedback from executions to determine the true value of experience and to continuously optimize memory is an important part of ongoing self-improvement.

The core challenge among them is Memory Credit AssignmentLong-term tasks may invoke multiple pieces of historical experience, but which of them actually influence the Agent's decisions, and which are merely read without having any effect? Even if the task is ultimately successful, determining the actual contribution of different memories remains an unresolved issue.

In response to this challenge, MemoraX willEstablish a connection between memory versions, actual usage records, and task execution feedback.Whether historical experience has actually influenced the actions of the Agent, as well as the task performance and validation results after its use. Based on this, continuously evaluate and iterate the existing memories.Retain and reinforce effective experiences, and promptly revise those that are no longer applicable based on feedback from new tasks.

These real feedbacks also contribute to...Continuous Training of Memory-Based ModelsThe data foundation and optimized signals provided enable improvements in memory to extend beyond mere content updates, to the memory model's learning of the value of experiences and memory strategies.

03 | From Experience Accumulation to Skill Improvement:

MemoraX's Multi-Scenario RSI Implementation

For Agent RSI, what is more important is not just how well a particular task was performed, but rather...Can historical experience be continuously accumulated and refined, and continue to generate value in subsequent tasks?

Around this process, the continuous iteration of experience, the ability to transfer knowledge across tasks, and the optimization through feedback were verified in three types of tasks: long-term complex coding, financial analysis, and complex mathematical reasoning. Additionally, it was further demonstrated how memory contributes to the improvement of the quality and efficiency of the Agent in long-term tasks.

(1) Long-Distance Complex Coding: From Experience Iteration to the Enhancement of Complex Engineering Skills

Long-term, complex coding experiments are conducted in fields such as communications, finance, and logistics. In 5 major industries and 13 industrial-grade system development projects, a total of 78 development runs were conducted, with a combined development duration of over 300 hours, with the longest single run exceeding 12 hours. As a result, the adoption rate of the MemoraX memory enhancement solution increased by more than 20%, and the cumulative Token consumption of the main Coding Agent was reduced by more than 20%.

From Experience Accumulation to Recursive Improvement: The Path of Self-Iteration in Memory

To address the issue of traditional static memory's inability to continuously accumulate and optimize experience during task execution, the MemoraX team has proposedMemory Model–Driven RSI (Recursive Self-Improvement) MethodBy coordinating with the Agent system using a proprietary memory model, the dedicated memory model is responsible for extracting and updating procedural memory from the Agent's execution traces and online feedback. The Agent system then applies this memory to actual tasks, creating a closed-loop iteration mechanism that includes experience extraction, task application, effect verification, and memory update. This allows for the continuous accumulation and optimization of effective experience, which in turn guides subsequent tasks. In training use cases consisting of long-term coding tasks,After multiple rounds of memory iteration and optimization, the quality of procedural memory has increased by more than 10%.

With the experience gained from repeated iterations, I took on the challenge of developing a complex project that lasted for 335 hours.

How much role can the accumulated experience play when it is separated from the original learning tasks and faced with new engineering challenges?

To this end, the experience already accumulated will be applied to the development of entirely new and complex engineering projects.Across 13 complex software engineering tasks, a total of 78 development runs were conducted, with a combined development duration of more than 335 hours, with the longest single run exceeding 12 hours.These tasks are far more than just simple code modifications; they involve complex business logic, cross-module dependencies, and a large number of tests, representing real engineering challenges.

Ultimately, the pass rate increased by more than 20%, and the total cumulative Token consumption of the Coding Agent also decreased by more than 20%.

Among them, AccessSentinel is a representative and challenging case. The Agent needs to build a comprehensive enterprise-level permission control system and ensure...Strict Security Constraints。

With the help of the memory model,Historical experience is no longer just a static reference; it can continuously participate in the execution and verification of the Agent as the task status changes.From 43 minutes after the development started until 2 hours and 30 minutes later, memory was recorded at different key points.Proactively offerTargeted Guidance:

• When device permissions are revoked，Not only should the device be prevented from logging in again, but it is also necessary to ensure that accounts that have already logged in cannot continue to access the system.
• When restoring the old version of the security policy,We need to maintain historical records while also preventing any previously revoked permissions from being accidentally reinstated.
• When processing applications in batches，First, check whether all applications meet the requirements. Only when all of them pass can the process be executed uniformly to avoid situations where some applications are successful while others fail.

During the testing,The average pass rate for the three runs increased by more than 45%, while the cumulative token consumption of the main Coding Agent decreased by more than 66%.

(2) Complex Financial Analysis: From Evidence Accumulation to Autonomous and Continuous Improvement

Financial research requires forming complete and reliable analytical judgments from a large amount of dispersed materials. In addition to finding relevant data, the Agent also needs to accurately understand the entities, timing, and statistical criteria within the complex context, and establish connections between different pieces of evidence to avoid affecting the final conclusions due to information omissions or confusion in the criteria.

MemoraX focuses on two interrelated issues:How to use evidence to support a comprehensive answer, and how to utilize the questions within the answer to improve the methods of evidence preservation and retrieval.To this end, MemoraX conducted experiments based on the authoritative financial benchmark FinRank, which covers the financial reports of 22 publicly traded companies in the United States from industries such as pharmaceuticals, oil and gas, and automotive. These reports include a large amount of complex financial data across companies and time periods. MemoraX uses task feedback to continuously optimize the organization and retrieval of information, forming an RSI (Recall, Search Similarity, Improvement) iterative loop that follows the sequence: "execution — feedback — memory improvement — re-execution." More than 30 collaborative iterations between the memory model and the financial analysis system, in the endThe evidence chain recall rate is close to 100%, which represents a 20% improvement in task completion accuracy compared to CodeX, and the cost has been reduced by nearly 98%.

From finding the data to organizing evidence that can support a judgment

Taking the financial analysis task of a large manufacturing company as an example, the evidence related to this task is scattered across a large amount of similar industry materials that include financial statements for three consecutive years, supplier disclosures, and risk descriptions. It is easy to find content that is related to the topic but does not belong to the target company or the reporting period simply by using keywords or semantic similarity. However, if only summaries are retained, the value units and the conditions under which the risks exist may be overlooked.

In response to this, MemoraX first converts the contextual material into structured memory, and then iteratively adjusts the way the memory is organized based on feedback from the answers. It is tailored for financial question-and-answer scenarios, gradually developing a system that takes into account various aspects.Precise values, aligned calibers, and traceable originsThe memory model output representation; during retrieval, these evidences are associated and called based on the current issue, and the inventory status, procurement structure, and potential risks are analyzed together. This organizational approach takes into account the requirements of financial tasks.Accurate values, consistent measurement units, and traceable sourcesThe requirements of .

The final task completion accuracy of MemoraX has improved compared to the Codex baseline.Nearly 20%In addition, based on the actual API usage statistics, the cost of this solution is...Compared to the Codex reference solution, it has reduced costs by nearly 98%, while achieving a higher pass rate.

From feedback on responses to strategy iteration: Improving memory through "how to save, how to retrieve"

MemoraX, based on feedback from tasks, continuously explores ways to select and organize information in memory in response to issues such as evidence omissions, relationship confusions, and redundant references, and continuously accumulates knowledge through training tasks.Interactive examples that demonstrate the relationship between writing decisions, the content stored in memory, and the subsequent effects of their use;Used to optimize the memory model. The trained model is then involved in a new round of tasks, and its execution feedback is used to further improve the writing strategy and subsequent training, creating an RSI (Reinforcement Learning with Interaction) cycle where task interaction and model optimization reinforce each other. In this process, the model gradually develops...Memory representation characteristics adapted to financial scenariosSave the key numerical values along with their applicable conditions, distinguishing between different entities, reporting periods, and sources. Establish a correspondence between the facts and the original evidence. After that... 36 rounds of RSI iteration，MemoraX The evidence recall rate reached 96.49%, and the task accuracy increased from 33.33% to 73.33%.

For researchers who need to track financial reports and announcements over the long term, the memories constructed from the same set of materials can be used to address different issues, thereby reducing the initial investment. Feedback from subsequent analyses can then be used to further refine the content of these memories and the strategies for their use.Make the answers more comprehensive, make the use more cost-effective, and enable continuous accumulation of experience,Together, they constitute the practical value of MemoraX in complex financial tasks.

(3) Complex Mathematical Reasoning: From Empirical Feedback to Self-Iterating Problem-Solving Strategies

In high-difficulty mathematical reasoning experiments, MemoraX further explored the use of historical experience and validation feedback to support Agents in continuously refining their reasoning and optimizing problem-solving strategies when dealing with complex issues. In combined optimization problems of the IMO (International Mathematical Olympiad) level, the Agent was able to... 4 hours of continuous solving, more than 600 model calls,Finally, the verified version has been found. The optimal solution for 21 rectangular blocks: Compared to the OpenAI Codex approach, the solution time has been reduced by 13%, and the cost has been reduced by 98%.

From Proof of Failure to Optimal Solution: Continuously Improving Reasoning through Feedback

Matilda 16×16 is, in my opinion, a highly challenging combinatorial optimization problem: it requires using the fewest possible rectangles to cover the 240 squares within a 16×16 grid, while ensuring that exactly one square remains empty in each row and each column. There are already quite a few possible solutions just for the empty squares alone. 20.9 trillion possible permutationsThe difficulty lies not only in finding a feasible solution but also in proving that there does not exist a scheme that uses fewer rectangles.

In the face of this challenge, the agents continued to collaborate on constructing search algorithms, formal proofs, precise modeling, and independent audits. The system initially identified 27 and 23 candidate constructs, but the argument for the global optimality of the 23 constructs was rejected by the independent audit.

MemoraX is capable of continuously accumulating previous candidate solutions, reasoning processes, and validation feedback. Its memory-based models extract effective improvement lessons from successful outcomes and transform failed experiences into references for subsequent adjustments to the problem-solving strategies:

• Memory self-correction: From wrong conclusions to memory correction. Independent auditing rejected 23 pieces of incorrect proof of optimality, prompting the Agent to re-examine its existing conclusions. The verification feedback that “a feasible solution is not necessarily the optimal solution” was used to correct the Agent’s memory, preventing subsequent reasoning from making the same mistakes again.
• Strategy Self-Optimization: From Memory Updates to Method Improvements. Based on the previous candidate solutions derived from the memory-based model and the validation feedback, the Agent was guided to adjust its modeling approach. The empty spaces and rectangular boundary constraints were uniformly incorporated into the search for solutions. As a result, 22 and 21 better solutions were found respectively.
• Memory Iteration: From Result Verification to Continuous Optimization. The agent achieves an accurate solution by conducting a step-by-step examination and using various tools. It ultimately confirms the optimality of the answer and feeds back the new verification results to the memory base model to enhance the construction of the solution memory, providing a reference for similar problems in the future.

The optimal arrangement of 21 rectangular tiles for the Matilda 16×16 problem. The black squares represent empty spaces, while the areas of different colors represent the rectangular tiles.

This demonstrates the Agent's continuous iterative capability in complex mathematical tasks:Historical experience is no longer just a static record of reasoning; it can be combined with new verification feedback to continuously contribute to the adjustment of subsequent problem-solving strategies.

Starting from memory, moving towards continuously evolving agents

MemoraX AI (Yiji Yuan) was founded by Hao Jianye, former director of Huawei's Decision and Reasoning Laboratory, head of the Large Model Algorithms Laboratory, and chief expert in the field of intelligent decision-making. It focuses on the development of the next generation of Agent memory models that are capable of learning and evolving.

The memory model independently developed by the team, based on Agentic RL technology, won the championship in a globally authoritative memory ranking and has received extensive coverage and attention from media both domestically and internationally, including the MIT Technology Review. The team's memory solutions have been widely implemented in leading enterprises across industries such as programming, healthcare, finance, security, and smart homes, and they are strategic partners of Huawei Cloud and Tencent Cloud.

Today, the team is further advancing the RSI (Risk Sensitivity Indicator) based on memory-based models, enabling Agents to move from accumulating and reusing experience to continuous learning and self-evolution, thereby solidifying their leading position in the field of AI memory. At the same time, they are accelerating the implementation of these models in various scenarios and promoting the standardized development and large-scale commercialization of memory modules for B2B (business-to-business) industries and C2C (consumer-to-consumer) products.

Conclusion

The key to Agent RSI is not just about making AI perform better in a single task, but also about...Let every execution become the foundation for the next improvement in capabilities.Successful methods, lessons from failures, and applicable conditions are accumulated through long-term memory and are transformed into reusable guidelines for action in new tasks; the results of these new implementations, in turn, drive the updating of experience, the optimization of memory strategies, and the continuous learning of the underlying memory models.

What MemoraX is exploring is...A bidirectional evolutionary path where memory and Agent self-improvement reinforce each otherOn one hand, this allows memory to provide a continuous foundation of accumulated experience that can be transferred across different tasks for Agent RSI; on the other hand, the execution and feedback in real tasks continuously contribute to the learning and evolution of the memory itself.

When every task leaves behind experience, and every piece of feedback drives improvement, the capabilities of the Agent are no longer limited to a single execution; instead, there is an opportunity for it to become a process of continuous accumulation and self-evolution.

The article is from "Machine Heart", written by "Machine Heart".