Researchers used Claude to break into OpenAI systems

Three researchers from Hacktron AI managed, during an authorized security test, to gain access to ChatGPT accounts belonging to OpenAI employees by using Claude to automate part of the operation. According to the report on the experiment, the entry point was an OpenAI forum hosted on Discourse. The researchers then reached ChatGPT accounts and information about the software infrastructure and reported the vulnerability through the bug bounty program. OpenAI fixed the issues. The case was covered in greater detail in an article published today on AIdapted.ro — AI vs. AI: Claude helped break into OpenAI accounts.

Why it matters: the experiment shows how much AI can reduce the time needed for a security operation. At the same time, the ability to rapidly analyze systems, find vulnerabilities and automate successive steps can also be used by attackers. AI security is therefore becoming different from ordinary software security: it is necessary to protect both the system itself and the ways in which AI can interact with it.

A chatbot told a patient “Congratulations!” when he spoke about assisted dying

An Australian parliamentarian reported during a parliamentary inquiry the case of a terminally ill patient who had written to him that he intended to pursue voluntary assisted dying. According to The Guardian, Microsoft Copilot suggested responses to the parliamentarian such as “congratulations!”, “great to hear from you” and “that is wonderful news!”. The case was presented during hearings on September 18 concerning the use of AI in Australia.

Why it matters: the problem is not simply an inappropriate phrase. When AI is used to draft responses about death, illness or crisis situations, a failure to understand context can have consequences far more serious than an ordinary factual error. The case shows why human supervision remains important when AI is used in extremely sensitive communications.

Anthropic is testing access to Claude for bank accounts

Anthropic is testing a feature called Claude Money, which would allow users to connect their bank accounts to Claude for spending analysis and financial planning. According to information published about screenshots of the application, the feature appears in an iOS version currently being tested. The compatible banks and the date of a public launch are not yet known.

Why it matters: the difference between an AI that explains your finances and one that can actually see your transactions is significant. If the feature is launched, Claude could analyze income, expenses and recurring payments, making it more useful while also giving it access to some of the user's most sensitive personal data.

OpenAI enters the market for AI tools for lawyers

OpenAI has launched Astra for Law, a specialized configuration of GPT-6 Astra for law firms and legal-tech companies. The system combines the model with a specialized index for legal research and instructions for legal analysis and drafting. According to Reuters, Astra for Law is initially being offered to selected law firms, while companies such as Harvey and Legora can use it for their own products.

Why it matters: specialized AI is increasingly being built for professions with their own rules, language and databases. In the legal sector, a system that can directly search case law and legislation could change part of the research and drafting process, but it does not eliminate the need for human verification — particularly in a field where an error can have direct legal consequences.

Research shows that AI can produce more convincing phishing than humans

A study conducted by researchers from Brigham Young University examined spear-phishing messages (fraudulent messages personalized for a specific individual) generated with AI. Research published by Phys.org shows that AI-generated messages can be more convincing than those written by humans because they can rapidly combine information about a victim's occupation, colleagues, hobbies or online activity.

Why it matters: one of the traditional ways of spotting fraud was to notice messages that were poorly written or too generic. AI removes precisely this weakness. A message can be fluent, mention a real colleague and contain an authentic detail from your life, without the sender actually being who they claim to be.

Customers are starting to ask for humans again when chatbots cannot solve the problem

A Gartner survey of 3,566 B2B and B2C customers found that 50% of respondents consider customer-service interactions easier when companies use AI. At the same time, 87% say it is essential to have the ability to reach a human agent. Gartner notes that users react negatively when AI becomes a barrier between them and an employee.

Why it matters: people do not appear to be rejecting AI itself. The problem arises when AI becomes the only option. A chatbot can quickly solve a simple question, but when an unusual, financial or personal situation arises, the user wants to be able to transfer the conversation to a human who can make a decision.

Anthropic says AI is already helping build the next generation of AI

Anthropic says Claude is already contributing significantly to the company's research and development activities. According to Associated Press, in August Claude was involved in approximately 26% of model research and development efforts, compared with 0% in February, while around 90% of R&D activities included collaboration with AI. Anthropic emphasizes that humans continue to supervise the process.

Why it matters: a new loop is emerging in AI development: one model is being used to build and improve the models that will follow it. This is not, at this stage, fully autonomous self-improvement, but if the share of work carried out with AI continues to increase, existing models could contribute more and more to the pace at which future generations appear.

An AI-powered attack compromised an organization in Spain and modified personal data

Spain's data protection authority, AEPD, received a notification concerning an attack in which an AI agent was reportedly used to search for vulnerabilities, gain access and then modify personal data and access invoices. Reports about the incident say the attack is still being investigated and that the identity of the organization, attacker and AI model used have not been made public.

Why it matters: the new element is the level of autonomy. AI was reportedly not used only to generate code or explain a vulnerability, but to carry out several stages of the operation. For companies, this means that defenses must account for attacks capable of rapidly analyzing a system and adapting their next step, rather than only attacks executed according to a fixed scenario.

Mistral and other European companies challenge the idea of slowing AI development proposed by US labs

French company Mistral and other voices in the European technology industry are challenging calls from some US laboratories to slow the development of advanced AI models. Reuters reports that European representatives argue that such measures could strengthen the position of already dominant US companies, while supporters of stricter rules argue that the current pace of development creates risks that require coordination.

Why it matters: for Europe, the debate has two dimensions that can come into tension: AI safety and technological competitiveness. Europe is trying to develop its own models and infrastructure in a market dominated by US companies, meaning that any proposal to slow development is also being examined through the lens of technological dependence.

The UK reopens the debate over a dedicated AI safety law

In the United Kingdom, pressure is growing for a stronger legal framework covering the safety of advanced AI systems. The Financial Times reports that a motion is due to be discussed at the Labour Party conference, while the UK minister responsible for AI has indicated that the government is open to additional legislation and stronger oversight if necessary.

Why it matters: the debate could move the UK from a framework based largely on voluntary rules and commitments toward clearer legal obligations. For companies and users, such a change could mean additional requirements for testing, reporting and accountability before certain AI systems are deployed widely.

The EU begins coordinating the enforcement of the AI Act more closely

The European Union's AI Board met on September 17 to discuss the implementation and enforcement of the AI Act, recent incidents, the Commission's supervisory activities and the development of European AI capabilities. According to the European Commission, discussions included frontier AI capabilities, safety, recent incidents, AI adoption in industry and coordination between member states. The Republic of Moldova participated for the first time as an observer.

Why it matters: the AI Act is increasingly entering the stage where the practical application of the rules matters more. For companies and users, this can affect how AI systems are evaluated, how incidents are investigated and how coordinated the responses of authorities across member states are.

Firefox integrates a European AI model from Mistral

Mozilla and Mistral have announced a partnership through which the Mistral Small 4 model is available in the Firefox Smart Window beta. According to Mozilla, the model is initially being introduced to users in the United States and Canada, while Smart Window beta and French-language support have also been expanded in France.

Why it matters: AI is beginning to become part of the browser itself, without requiring users to open a separate chatbot. For Firefox, integrating a European model also means offering an alternative to the models developed by major US companies. For users, however, it is becoming increasingly important to know which model is processing their information and how much context from their online activity is being sent to it.

Sources: