When the computer/AI says no, government still has to explain why

A resident applies for assistance, a permit, a benefit or another government service. The computer reviews the information and rejects the request. while the answer arrives quickly, what may not arrive is an explanation.

As public organizations turn, including St. Maarten's government apparatus, seek to turn to artificial intelligence and automated decision-making to reduce workloads and speed up services, they must consider that a new problem has already emerged: a decision can appear efficient while leaving both the resident and the public servant unable to understand how it was reached.

This is the concern raised by Anne Bonvanie, a researcher in ethics and technology at Saxion University of Applied Sciences in the Netherlands. Writing for iBestuur, Bonvanie argues that the use of artificial intelligence in government must be accompanied by stronger attention to equality, transparency, care and professional judgment.

The issue is not whether government should use technology. Used properly, automated systems can process information faster, reduce repetitive work and help residents receive answers without unnecessary delays. The danger begins when speed is treated as proof that the system is fair.

Technology does not make decisions on its own

Artificial intelligence is often presented as neutral. A computer has no personal grudge, political loyalty or bad mood. It appears to simply review the information and produce an answer. However, the system did not create itself. People decided which information it should use, which patterns it should search for and which factors should carry the most weight. Historical records were selected to train it. Rules were written. Categories were created. Every one of those steps involves human choices.

When historical data contain unequal treatment or bias, an automated system can learn and repeat those patterns. It may even apply them more consistently and on a much larger scale than any individual employee could. Bonvanie points to major Dutch controversies involving childcare benefits, student financing and welfare-fraud investigations as examples of what can happen when biased assumptions are translated into algorithms or risk-selection systems. The lesson is simple: a computer can apply a bad rule faster, but that does not make the rule better.

The danger of trusting the screen

A public servant who receives questionable advice from a colleague is likely to ask questions. Why was this person selected? What information supports the conclusion? Was something important overlooked? The same level of caution may disappear when the recommendation comes from a computer. The system appears scientific, consistent and free from emotion, so its output can be accepted with less scrutiny.

Bonvanie warns that this can make the values behind a decision almost invisible. An employee may see only the final recommendation, without knowing which assumptions or data led to it. That creates a serious accountability problem.

A resident cannot properly challenge a decision when government cannot explain it. An employee cannot correct the system when the employee does not understand how it works. A minister or department head cannot assure the public that people are being treated fairly when officials themselves have limited insight into the technology. “Computer says no” cannot become the final response of government.

St. Maarten context

For a small country such as St. Maarten, digital systems offer clear benefits. Government departments face staff shortages, growing service demands and public frustration over waiting times. Technology can help process applications, organize records, identify missing documents and direct routine cases to the correct department.

Artificial intelligence may also help employees review large amounts of information and respond more quickly to common questions. But St. Maarten’s size also creates particular concerns. People’s circumstances do not always fit neatly into standard categories. Families may have complex living arrangements. Employment may be temporary or spread across different sectors. Documentation may be incomplete. Residents may move between the Dutch and French sides or have personal and economic connections across several Caribbean islands.

A system designed around rigid assumptions may misread those realities. Someone who fails to submit information in the expected format may be treated as ineligible rather than referred to an employee for assistance. A resident with an unusual employment history may be identified as suspicious. A family whose circumstances fall outside the normal pattern may be rejected because the system was trained to recognize only the most common cases. The computer may process every case in the same way while still producing unequal outcomes.

Efficiency can hide unfairness

Consistency is often presented as one of the strengths of automation. Two people who submit the same information should receive the same result. That sounds fair, but equal treatment does not always mean repeating the same process without examining individual circumstances. A missing document may be the result of carelessness, but it could also reflect illness, displacement, language difficulties or an emergency. A late response may indicate neglect, or it may mean the person never understood what was requested.

Human judgment allows an employee to recognize those differences. An automated system may simply record that the requirement was not met. This is where efficiency can become dangerous. The process appears successful because decisions are being issued quickly, backlogs are falling and fewer employees are required. At the same time, vulnerable residents may be excluded before anyone notices a pattern.

Bonvanie describes a situation in which an AI tool handles many applications correctly but rejects a vulnerable group too quickly because its members do not provide information in the format the system expects. The appropriate response is not necessarily to discard the technology, but to adjust it so unclear or incomplete cases are automatically sent to a human employee. That is the kind of safeguard government systems require.

Governments often promise that a person will remain involved in automated decision-making. That promise means little when the employee is expected to approve the computer’s recommendation without enough time, knowledge or authority to challenge it. Real human oversight requires trained employees who understand what the system does, know its limitations and are permitted to reject its conclusion.

They must also be able to explain a decision in language the resident can understand. This changes the role of the public servant. Employees will increasingly work with automated systems rather than separately from them. Their job will not only be to use the output, but to test it, question it and recognize when it does not fit the facts. Government must create room for that professional judgment. It cannot introduce technology to save time and then deny employees the time needed to examine questionable results.

Questions government should answer

Before an automated system is used to influence a permit, benefit, inspection, tax matter or social-service application, the responsible public body should be able to explain its purpose. Officials should know what information the system uses, who developed it, what risks were identified and how errors can be corrected. They should also know whether certain communities are being rejected or flagged more often than others.

Residents must have a clear route to request human review. These protections should not be added only after people are harmed. They must form part of the system from the beginning. Outside technology providers also cannot be allowed to hide important decisions behind commercial secrecy. Government remains responsible for the outcome, even when the software was built by a private company.

The final responsibility remains human

Artificial intelligence can assist government, but it cannot carry government’s moral or legal responsibility. A computer does not face the resident who loses access to assistance. It does not explain itself in Parliament. It does not answer questions from an ombudsman, court or community. People do.

The real test of public-sector technology is therefore not how many applications it processes or how much money it saves. The test is whether it helps government serve residents fairly, accurately and with explanations that can be understood and challenged. St. Maarten should embrace technology where it improves public service. It should be equally prepared to slow the process down when a person’s rights, livelihood or access to essential support is at stake.

The computer may recommend no.

Government must still be able to explain why.

‍