The LegitiMate

A first attempt to in-depth research on the development and use of ADM in Dutch public administration.

MinBZK Guide
Definitive version

This version:
https://open-regels.nl/lm/thelegitimate/1.0.0
Latest published version:
https://open-regels.nl/lm/thelegitimate/
Latest editor's draft:
https://minbzk.github.io/LegitiMaat/
Editors:
(BZK)
(Hooghiemstra & Partners)
Authors:
Marlies van Eck
Mariette Lokin
Maike Klip
Giulia Bössenecker
Carlijn Oldeman
Robert van Doesburg
Abram Klop
Steven Gort
Participate:
GitHub MinBZK/LegitiMaat
File an issue
Commit history
Pull requests

This document is licensed under a Creative Commons Attribution 4.0 License.

Abstract

Public administration has many different public tasks. These tasks are assigned by a specific act to a specific public agency. Some of these legal tasks are automatically executed. In those cases algorithms are used. We call it ADM, Automated Decision Making. This is the process leading to legally binding results; administrative decisions in individual cases. ADM is used to impose fines for speeding, reclaim allowances or to grant state pension or child benefits. As with a more visible performance of public tasks such as building dikes, it is important to know how these activities contribute to the designated task and which choices have been made. Automated execution of legislation is a public activity too. It must therefore be legitimate and open for review.

The LegitiMate is an assessment tool to provide insight into and assess the automated execution of legislation by the public administration in a standardized way. The LegitiMate is based on three perspectives: legal, development and audit/ accounting perspective.

The project is the first attempt to get a common understanding of the daily and sometimes long existing use of algorithms by public administration. We are talking about systems that are programmed in COBOL. We understand that the word “algorithms” nowadays is often used in relation to Machine Learning (ML). This is however not the focus of this instrument. The LegitiMate is designed for knowledge-based algorithms (preprogrammed) used for automatically execution of legislation leading to an individual administrative legal decision.

The scope of The LegitiMate

The LegitiMate covers the entire process from the publicantion of an Act to an individual decision, the response of the recipients (phone calls, complaints, objections) and the chain effects (the transfer of the decision as data to third party decision recipients). It is necessary that this process becomes more transparent. Another aim of the The LegitiMate is to enable research or judgements on whether the legislation, the general principles of good governance and the prohibition of discrimination have been complied with.

The development of The LegitiMate

Experts developed The LegitiMate on behalf of the Ministry of the Interior and Kingdom Relations. This project was experimental and made with support from two public agencies that are using ADM for years, National ombudsman and several governmental institutions with knowledge of IT development. In the slipstream of the project scientific research was facilitated on related topics for master students.

Elements of the LegitiMate

The LegitiMate contains an analysis of the usual steps taken in public agencies. This is the “process map”.

Based on the process map we constructed “The LegitiMate List of Documents”. An inventory of documents required for the investigation.

Next we produced 3 questionnaires that are more or less based on the several steps mentioned in the “process map”. They are designed for each of the 3 perspectives. The questionnaires can be answered by the agency itself.

The third element is a description of the necessary steps to conduct the visitation research based on the information and answers.

With The LegitiMate we bring daily practice, technology and law one step closer together.

Want to know more?

For information about The LegitiMate, you can go to: A cool experiment in NL: The LegitiMate. Working method to judge the use of algorithms in ADM from 3 perspectives. – Automated administrative decisions and the law

Status of This Document

This is the definitive version of the guide. Edits resulting from consultations have been applied.

1. The LegitiMate

This section is non-normative.

1.1 Introduction

Public administrations have many different tasks. These tasks are assigned by an act to a specific public agency. Some of these legal tasks are automatically executed. In those cases algorithms are used. For example, imposing fines for speeding, reclaiming allowances or granting state pension or child benefit. As with a more visible execution of public tasks such as building dikes, it is important to know how these activities contribute to the designated task and which choices have been made. Automated execution of legislation is a public activity too and must therefore be legitimate.

While in literature the concept of “digital discretion” is elaborated, in practice Dutch public administration isn’t obliged to show and tell how the buzzing IT-systems are designed and how the design and use is in accordance with the legislation, principles of good administration and the prohibition of discrimination. This explains why up to today public agencies are barely aware of the legal and other aspects of these routines. With The LegitiMate, the executive branch of the government can be held accountable. It facilitates as well that external reviewers can form an opinion of the choices and system the public administration has (let) developed and implemented. The LegitiMate is intended to assess in a way that the administration can develop and improve. It is a visitation research tool.

1.1.1 Who is The LegitiMate for?

The LegitiMate is a method to understand and assess the automated execution of legislation by public administration in a standardized manner. The LegitiMate is based on both the highly specialized professionalism of the employees of the administrative agencies, and the increasing demand in society for public administration to be accountable for ever growing automation.

It is intended for public agencies that automatically execute legislation and make individual administrative decisions (ADM). With The Legitimate the agency can show the internal productions, the relevant steps and safe guards. It will probably lead to self-assessments and better understanding at management level of the range of responsibilities. The agency can also invite a visitation team with assessors from other public agencies to conduct research. With the result, the agency is provided with recommendations.

When new IT systems are being developed, the agency can immediately see what documents are needed in order to provide (external) insight.

The LegitiMate is also intended for assessors: the lawyers, auditors and fellow IT developers who need meaningful information on how the legislation is automatically executed.

1.1.2 How can I contribute?

On July 1, 2022, the Dutch prototype was presented to the Ministry of the Interior and Kingdom Relations and published on GitHub. The working method has been developed, tested and adapted in a few sprints. Please contact Marlies van Eck if you want to contribute or experiment with the use of The LegitiMate.

1.2 A method to evaluate and learn

The LegitiMate is based on the principles of visitation research. It is a visitation tool. Large parts of the research are delegated to the expert, the public agency.

The LegitiMate indicates the necessary documents; The LegitiMate List of Documents. Another part of the LegitiMate are the three questionnaires that need to be answered by the administrative agency: The LegitiMate Questionnaires.

Once the answers and documents are collected, the visitation team will start its investigation by conducting interviews and audit sampling.

The LegitiMate is an important first step towards transparency and accountability for the use of algorithms in automated execution of legislation.

1.2.1 Interdisciplinary and generic

Auditors and assessors each have their own professional frame work and focus points when they want to assess the automated execution of legislation. We figured that it should be possible to align at least three disciplines to get a more holistic approach. On the other hand we try to help public agencies by standardizing the set of documents and questionnaires so they are not all asked to invent the wheel.

The LegitiMate is meant as an holistic method facilitating three disciplines in assessing automated execution in different types of public agencies.

With The LegitiMate:

  • a legal expert can judge the legitimacy of the automated execution including compliance with the right to good administration and prohibition of discrimination,
  • an information expert can judge the quality of the automated implementation,
  • an accountant or internal controller can judge the financial legitimacy of the automated execution.

1.2.2 Relationship with other initiatives

In the Netherlands a lot is happening regarding the use of algorithms by the public administration. For example, members of the Parliament voted in favour of a National Algorithm register for public agencies and the government has an open source policy.

Also, increasing attention is paid on the supervision of the use of algorithms. However, supervisory activity is only possible if the public agency itself has insight into its own complex and highly technical (legal) processes.

1.3 How is legislation automatically executed?

When public administration makes administrative decisions in individual cases involving large numbers and many comparable repetitive tasks, this process often is automated. To enable this the particular act/legislation needs to be translated and choices must be made to decide how the legislation will be implemented. After discussions with several experts in different agencies working on these translations and implementation we distinguished this process in the following steps.

Processen wetsuitvoering
Figure 1 Processen wetsuitvoering

This process map is an illustration, not a prescription. All public agencies have their own practices, internal traditions and work flows. This will remain. The process map and the several stages we describe and use in The LegitiMate are a means to an end: a dialogue.

1.3.1 Automated decisions or decision support systems?

The LegitiMate is designed to assess the automated execution of legislation by public agencies. In literature on ADM its very common to distinguish easy from hard cases. This difference can be witnessed in practice too. To understand the internal routines its necessary to discuss the commonly used difference in easy and hard cases.

Easy cases are processed without human intervention: the decision is made automatically.

A minority of cases is partly handled with human intervention (hard cases / /dropouts/ejected). In these cases the algorithms perform as decision support systems.

The public administration uses selection or prioritization methods to distinguish cases in those who lead to fully automated decisions and those that need human judgments and therefor are partially automated. By applying selection rules (sometimes multi staged / in several rounds to match available staff capacity), the case can be assigned to a public servant for processing. The latter may request additional information or carry out additional checks resulting in a decision made in collaboration with the machine. In the legal perspective, we call this the preparatory acts.

Voorbereidende handelingen
Figure 2 Voorbereidende handelingen

1.3.3 On algorithms and source code

Ultimately, algorithms are converted into computer code: the source code.

1.3.4 The scope of The LegitiMate

The LegitiMate covers the entire process from act to individual administrative decision and afterwards, the response of the citizens (telephone calls, complaints, objections) and the chain effects / propagation.

It is made for the automated execution of legislation by the public agencies using knowledge-based algorithms.

Knowledge-based algorithms use rules for making calculations, solving a problem, answering a question, or making a decision. The rules we are talking about can be divided into different types: decision rules (with which you test variables for true/false, and then arrive at a decision) and calculation rules (with which you make a sum with those variables). There are many more, although they may also be classified as subcategories of these two.

If it has to be determined whether an applicant falls under a certain category (e.g. “partner”), you also run through a number of variables in the definition of partner, and actually make a decision about that. After this you take the result of this into account in a subsequent decision (e.g. about entitlement to a supplement). We could also call that a classification rule.

Selection rules determine which decisions must be taken (partly) by a human civil servant. Sometimes these are rules are preprogrammed (such as 'all cases in which applicants provide an address other than that is officially registered) and sometimes statistical algorithms are used.

The LegitiMate therefore isn’t (yet) suitable for accounting for or assessing the use of statistical algorithms, including machine learning that make taxations about the probability of an outcome based on statistical analysis of data from the past. This may be addressed later.

1.3.5 Propagation

Within public administration, many personal data and systems are interconnected. This means that both the input (data) and the result (a decision) have relationships with other processes, other domains, other laws and other agencies. Providing insight into these relationships and assessing whether this implementation is in accordance with the law, principles of good administration and the prohibition of discrimination is also part of The LegitiMate.

1.3.6 What about IT-systems bought by the public administration?

A lot of documentation is needed to assess the legitimacy of the systems. Public administration is responsible for the correct interpretation of legislation and compliance with general principles of good administration. This doesn’t alter if it is not developed by public servants but purchased or commissioned. In case the documentation cannot be made available by the supplier or if the supplier refuses to do so due to intellectual property rights, The LegitiMate cannot be used. It will of course have implications regarding the accountability of the public administration.

2. The LegitiMate as a visitation instrument

2.1 Visitation

In visitation research learning and developing is the primary focus instead of a check or audit based on supervisory authority. It exists of tools like a self-evaluation and peer appreciation or assessment.

Visitations are used in all kinds of sectors where professionals are not exactly prescribed how to work but expected to work from professional insights, for example the medical world and education. In developing the visitation aspect of The LegitiMaat, we drew inspiration from several of these assessment systems.

2.2 How does peer visitation works?

Various scenarios for peer visitations can be found on the web; we drew from a number of them for The LegitiMate.

In principle, a visitation consists of three phases. The first phase is a self-evaluation by the organization, the second phase is a visit by an independent assessment team that speaks with those involved based on the results of the self-evaluation. Based on this, the team values ​​the organization and provides recommendations for further improvement. In the third phase, the organization implements the recommendations.

A visitation is periodically repeated. A three-year cycle seems to be the most effective and efficient. This gives room to careful preparation and implementation of the recommendations. Depending on experience, the three-year cycle can be adjusted.

The assessors in the independent assessment team are preferably civil servants of another public agency, who are directly involved in the development and application of algorithms in their organization. They can ask critical questions based on their knowledge and experience. Moreover, they may learn from the working methods and experiences of the organization they visit. It therefor works two ways. Regarding the objectivity of the assessment team, an independent chair and secretary will be provided (ie, who do not work for the agency that is reviewed/assessed.

2.3 The three phases

2.3.1 Phase 1 - Self-evaluation

The organization conducts the self-evaluation based on The LegitiMate List of Documents. and The LegitiMate Questionnaires. A coordinator divides the tasks within the assessed agency, monitors progress and checks supplied input for clarity, etc.

The start of the self-evaluation depends on when the assessment team can make the assessment visit. It can be taken as a guideline that the self-evaluation must be completed approximately four weeks before the visit of the assessment team (see timetable in section 4). This gives the team time to prepare for the visit. It is important that at least the chair and secretary of the assessment team are known at the start of the self-evaluation. This enables agreement and planning (in an introductory meeting) on deadlines.

2.3.2 Phase 2 - Visit visitation team: proof of the pudding

The visitation itself is carried out by the visitation team. The visitation team consists of the assessors with the three perspectives: legal, audit perspective and IT development perspective. We suggest 4 to 5 assessors and an independent chair and secretary. The visitation team examines to what extent the image of the organization about itself regarding the quality of the development and deployment of algorithms, corresponds to the image that the assessment team obtains based its own preparation (going through the self-evaluation and additional research).

While phase 1 focuses on obtaining documents and information about the processes, this part of the research focuses on the substantive / material outcome. Even well organized and documented processes are no guarantee that the result, the decisions and the legal protection or chain effects, are substantively in accordance with the law, the principles of good administration and the prohibition of discrimination.

Compare this to a driving exam; it can be checked whether the learning driver spots the mirrors and makes the necessary movements such as looking over the shoulder. The procedure can then be assessed with a ‘good’. However, this doesn’t mean the person actually sees what happens, recognizes the possible dangers in time and acts accordingly. You can only observe this by watching in practice and experience what goes right and what (almost) goes wrong.

How this part of the visitation research can best be designed depends on situation and available time. We recommend that at least the following is investigated:

  1. Case law: what are the most common disputes in the field of the chosen act and processes and corresponds the resulting image to the self-evaluation?
  2. Decision rules: if it is guaranteed that the decision rules form the basis of the code, it is advisable to investigate these. If the number is too large, tools to perform analysis can be tried (such as e-discovery tooling).
  3. A sampling of the administrative decisions (without and in combination with objection/complaint procedures) to judge the legality.

It is also conceivable that other methods can be used, using data analysis and/or process mining or decision-mining. Future experiments with these are on our wish list.

In addition, conducting interviews with public servants is essential. This concerns employees with roles and functions selected by the members of the visitation team.

No fixed standard is given for the time required; Depending on the size and structure of the organisation, one or more days can be scheduled for the visitation.

After the visit (or visits), the visitation team prepares a preliminary report with findings and recommendations on all components of The LegitiMate. Recommendations based on the findings are explicitly focused on the positive points (appreciation) and improvement (learning) aspects.

The visitation team discusses the report with the organization's MT. For example, an oral explanation can be given and the organization can respond (principle of hearing both sides). The report is finalized.

2.3.3 Fase 3 - Uitvoering van de aanbevelingen

De organisatie maakt een implementatieplan voor de gedane aanbevelingen. Hierin worden acties voor uitbouw of bestendiging van positieve punten opgenomen en acties ter opvolging van verbeterpunten. De acties worden van een concrete realisatietermijn voorzien, zodat zij in een volgende visitatie kunnen worden meegenomen.

2.3.4 Phase 3 - Implementation of the recommendations

The public agency designs an implementation plan. This includes actions for expanding or consolidating positive points and actions to follow up improvement aspects. The to-do-list is provided with a concrete realization period, so that they can be included in a subsequent visitation research.

2.4 Time Table

The table below contains an (indicative) timetable for the assessment process. The time margins are calculated from the moment of the (first) visit of the visitation team

Activity Indication of time Initiator
Introductory meeting with public agency 6 weeks in advance Chair & Secretary
Distribute The LegitiMate List of Documents and Questionnaires 6 weeks in advance Visitation coordinator of the public agency
Complete team members of the Visitation Team and share their names with the public agency 5 weeks in advance Chair & Secretary
Collect and check input on The LegitiMate List of Documents and Questionnaires 4 weeks in advance Coordinator
Share input with Secretary Visitation team 4 weeks in advance Coordinator
Design program of the visit of the Visitation Team and provide parameters case sampling 2 weeks in advance Secretary & coordinator
Check on program of the visit 2 weeks in advance All members of Visitation Team
Planning the interviews 1 week in advance Coordinator
Visitation Day; introduction, interviews, presentations, aftertalk Visitation Day Visitation Team
Written input for draft Visitation Report (by mail) 1 day after visitation Visitationteam
Drafting Visitation Report and send it to members of Visitation Team 1 week after visitation Secretary
Receive and process input members of the Visitation Team 2 weeks after visitation Secretary
Approve Visitation Report 2 weeks after visitation Visitationteam
Send Visitation Report to agency 3 weeks after visitation Chair
Plan conversation with agency 3 weeks after visitation Secretary & Coordinator
Conversation with agency 3 weeks after visitation Visitationteam
Draft the Implementation Plan 4 weeks after visitation Coordinator

3. Who and what?

If you want to get started with De LegitiMaat you need:

3.1 Which representative choices?

Even after the selection of a process (which act is automatically executed?) it is necessary to demarcate. Investigating the entire implementation process is usually not realistic, therefor it is advised to opt for a research that is sufficiently representative.

One method to get a representative image is to focus the research within the scope of De LegitiMate on two extremes:

  1. relatively easy decisions to make (determining age, for example)
  2. decisions that are relatively difficult to execute (with more room for discretion, such as a right or exception that must be determined based on facts and circumstances).

3.2 The assessors

The assessors are people who conduct the research based on their expertise. The LegitiMate is designed for three perspectives. Therefore it is recommended that the visitation team consist of at least an auditor, lawyer and a peer IT developer. Each expertise has a specific questionnaire. The method involves first obtaining an overall image, then three different in-depth investigations and finally an overall assessment.

The LegitiMate is developed as a visitation instrument. This means that the assessors can be employees of the organization itself (phase 1) as well as external experts who are members of a visitation team (phase 2).

3.3 The interlocutors

In order to have the right conversation, it is necessary to have conversations with public servants who fulfill a certain role. We talk about roles instead of function names because we experienced function names vary a lot or don’t have the same content. Both public servants who can explain the entire process and overall routines, and experts who can talk to one of the three types of assessors are needed.

3.4 The entire process and overall routines

The public servant responsible for developing and managing the IT-systems that automatically execute the act.

The public servant who knows how the automated execution of the act is -legally and tehnically- connected to the automated execution of acts by other public agencies.

The public servant who knows how responses of citizens who receive decisions are used to improve the automated process.

3.4.1 Developers perspective

The author of the package of requirements and/or functional design [link to glossary].

The software developer who writes code based on a set of requirements and/or functional design.

Administrator who manages the machine and application(s) on which/with the control service(s) run.

Manager dealing with Information Technology Infrastructure Library (ITIL) related matters.

3.4.3 Auditors perspective

The internal controller who checks whether the execution is financially legal.

4. The LegitiMate List of Documents

At the start of the research it is necessary that a series of documents is available for the assessors. We have called this The LegitiMate List of Documents. This may be quite difficult, so start early. All these documents are essential to conduct the investigation:

  1. The text of the act/legislation that is automatically executed.
  2. The policies of the administrative agency
  3. The set of decision rules that refer to the two extremes as indicated above.
  4. The analysis (if applicable).
  5. The functional design that refers to at least the two extremes.
  6. The interaction screens (screenprints) of the website that provides the information to applicants and facilitates the interaction.
  7. The letters citizens receive after their application (the administrative formal decision).
  8. The internal instructions describing the process and (manual) interventions/routines.
  9. Internal quality standards / requirements.
  10. Information on reported topics (calls, complaints or objections by citizens) and the numbers (preferably over several years).

6. Developers Perspective Questionnaire

6.1 Analysis

  1. Is there a detailed description of the current functioning of the (primary) process, where relevant, a description of the problem to be solved and its scope? Is the problem owner (the person responsible for the problem area) known?
  2. What should be achieved with the proposed change, if possible expressed in terms of improvement of legitimacy, efficiency, citizen satisfaction and employee satisfaction?
  3. Which user and/or policy scenarios are considered (sufficiently) representative to asses whether all principles / preconditions / principles / etc. are met? Take into account the whole range of the application and how the project promotor wishes to communicate about this: both in the solution(s) itself (ie business operations) and about the solution(s).
Want to know
Asses scenarios combined with communication policy. This is an example
A.
B.
C.
D.
E.

6.2 Decision Rules

Describes the mapping of the entire set of decision rules to the asses scenarios formulated in the analysis. It is clear for each scenario which decision rule set(s) is relevant for that specific scenario.

Want to know
Mapping asses scenarios on decision rules set. This is an example
A. mapt op set 1.
B. mapt op set 1.
C. mapt op set 1.
D. mapt op set 2.
E. heeft geen mapping.

6.3 Functional design

Special attention is paid to the place(s) where the functionalities with which the user and policy scenarios are supported and implemented in the analysis above are described.

  1. Is that (it/a) Functional Design? Or does the administration has another routine?
  2. Which FDs does the administration know? Examples:
  1. How are changes compared to previous version(s) tracked?
  2. If decisions are made that affect the functional design, that decision is preferably documented. Is that visible?
  3. Describes the mapping of the entire set of test cases to the decision rules. It is clear for each decision rule set which test cases are relevant for that specific decision rule set.
Want to know
Mapping decision rules on Asses scenarios. This is an example
Set 1 has test cases 01..07.
Set 2 has test cases 08..11.
Test scenario E lacks test cases because there is no fit with the decision rules set.

6.4 Code

Describes the used Git workflow for CI/CD and the corresponding version management of asses scenarios, decision rule sets and test cases. Also explains what the used test strategy is. Has the KPIs / reporting items of the process assessment and testing (BiSL).

Want to know
1. Version control
2. Test cases
3. Decision rules set
4. Assess scenario
5. Git workflow
6. Test strategy		 Not yet available

6.5 Digital Interaction

Explains separately for each individual assess scenario how the digital interaction takes place and which internal working instructions.

Want to know
Workflow per assess scenario Not yet available

6.6 Administrative Decision: the letter

During the analysis, communication policy has been established for and about(!) the solution(s). Here the connection is made with the way in which this has been processed in fixed and/or free text(s) for all test scenarios.

Want to know
Fixed vs free text(s) for all assess scenarios Not yet available

7. Auditors perspective Questionnaire

  1. How is algorithm management organized? Which departments are involved in the process of developing or modifying an algorithm?
  2. Who is the 'owner' of the algorithm within the administration: who has primary control over the development of new algorithms and changes to existing algorithms?
  3. How is the development or modification of algorithms in the manufacturing process documented (recording traces to sources, choices made, interpretations, etc.)?
  4. How is the algorithm development process recorded and documented?
  5. How was the choice for development methodology and supporting applications made?
  6. Has a central repository (repository) for algorithms been set up and how is it managed?
  7. Are the algorithms tested to ensure financially legitimate decisions and how? (LINK glossary)?
  8. How are the algorithms handed over from developer to owner? How is discharge granted for the correct execution of the development or change assignment?
  9. What architectures include the development and modification of algorithms and what procedures enable a compliancy check.
  10. How are the results of the application of the algorithms recorded for the purpose of evaluating (side) effects?

8. Begrippenlijst

ABBB de Algemene beginselen van behoorlijk bestuur (**on)geschreven regels waar de overheid en ambtenaren zich aan moeten houden bij handelingen en besluiten.

Algoritme Een procedure of reeks regels die worden gebruikt bij het berekenen en oplossen van problemen; een nauwkeurig gedefinieerde reeks wiskundige of logische bewerkingen voor de uitvoering van een bepaalde taak.

Algoritmeregister Een voorziening waarin een verzameling van algoritmen die worden gebruikt, openbaar wordt gemaakt.

Auditor iemand die zich beroepsmatig bezighoudt met het controleren van een organisatie volgens een vaste methodiek.

Awb Algemene wet bestuursrecht. Een wet die de algemene regels bevat voor de verhouding tussen de overheid en de burger.

Beleidsregels Op grond van artikel 4:81, eerste lid, van de Awb kan een bestuursorgaan beleidsregels vaststellen met betrekking tot een hem toekomende of onder zijn verantwoordelijkheid uitgeoefende, dan wel door hem gedelegeerde bevoegdheid.

Beoordelingsruimte Omdat er eigenlijk nooit voor alle gevallen precies bij wet is voorgeschreven wat er moet gebeuren, heeft een bestuursorgaan doorgaans beoordelingsruimte bij het toepassen van algemene regels in individuele omstandigheden. Het bestuursorgaan maakt van deze ruimte gebruik in de uitvoering, dus ook in een geautomatiseerde uitvoering.

Bereik van de LegitiMaat De LegitiMaat ziet op het gehele proces van wet tot aan individueel besluit en daarna, reactie van de ontvangers (telefoontjes, klachten, bezwaren).

Beslisambtenaar De ambtenaar die bevoegd is te beslissen in een individueel geval.

Beslisregels de waarmee variabelen worden getoetst op waar/niet waar, waarna het tot een beslissing komt.

Broncode (source code) de originele tekst waaruit de software voorkomt. Dit is een voor programmeurs leesbare tekst en bevat de programma-instructies over de werking van de functionaliteit van de code.

Decision mining techniek waarbij aan de hand van de resultaten, de besluiten, kan worden geanalyseerd langs welke routines de besluiten feitelijk zijn genomen.

Discretionaire bevoegdheid Bij een discretionaire bevoegdheid heeft de wetgever aan het bestuursorgaan een bevoegdheid verleend om in tot op zekere hoogte zelf te bepalen welke besluit wordt genomen (beleidsruimte). Sinds 2022 toetst de bestuursrechter het gebruik van deze ruimte bij een belastend (nadelig) besluit voor de burger indringender aan het evenredigheidsbeginsel van 3:4 van de Awb. In de context van geautomatiseerde uitvoering van wetten wordt wel gezegd dat dit vooral geschikt is voor wetgeving met gebonden bevoegdheden. Dit betekent dat uit de wettelijke bepalingen voortvloeit wat het bestuur moet besluiten.

Functioneel ontwerp hierin worden alle eisen en wensen waaraan een product moet voldoen verzameld en geordend. Er wordt beschreven op welke manier, welke verwachten de gebruiker heeft, welke handelingen hij uitvoert en welke resultaten die oplevert.

Information Technology Infrastructure Library (ITIL) een referentiekader ontwikkeld voor het inrichten van de beheersprocessen binnen een ICT-organisatie.

Intercoder reliability In de context van kwantitatief onderzoek is intercoder reliability de mate waarin 2 verschillende onderzoekers het eens zijn over het coderen van dezelfde inhoud. Het zorgt ervoor dat wanneer meerdere onderzoekers een set gegevens coderen, ze tot dezelfde conclusies komen.

Interdisciplinair het integreren van inzichten uit meerdere vakgebieden (disciplines).

Interne controller medewerker die belast is met het monitoren en controleren van bedrijfsprocessen.

Keteneffecten Binnen de overheid worden veel data en systemen aan elkaar verbonden. Dit betekent dat zowel de invoer (data) als het resultaat (een besluit) relaties hebben met andere processen, andere werkvelden, andere wetten en andere instanties. Het inzicht bieden in deze relaties en beoordelen of deze uitvoering volgens de wet, ABBB en het verbod van discriminatie is, hoort ook bij De LegitiMaat.

Machine learning Een machine learning (ML) algoritme moet worden getraind met een dataset. Het trainen houdt in dat het ML algoritme deze dataset analyseert en leert patronen en correlaties in deze dataset herkennen. Na het trainen, is het ML algoritme in staat de geleerde patronen en correlaties toe te passen op een nieuwe (onbekende) dataset. Doorgaans moeten ML algoritmes op grote hoeveelheden data getraind worden voordat zij accurate patronen en correlaties kunnen herkennen.

Open source beleid beleid waarin is bepaald dat de broncode vrij beschikbaar is. Iedereen kan de broncode lezen, aanpassen en verspreiden.

Process mining techniek om met behulp van event logs in applicaties processen te visualiseren en te analyseren. Het is een objectief, op feiten gebaseerd hulpmiddel is om processen te analyseren en te verbeteren.

Procesvertegenwoordigers mensen die het bestuursorgaan vertegenwoordigen in beroep of hoger beroep bij de bestuursrechter.

Rechtmatigheid het is bij de overheid gebruikelijk om een onderscheid te maken tussen juridische rechtmatigheid en financiële rechtmatigheid in het kader van controle-en verantwoording. Het juridische begrip rechtmatigheid gaat over alle geldende wetten en regels. Er wordt beoordeeld of het handelen in overeenstemming is met deze wetten en regels. Het begrip rechtmatigheid dat door de accountant voor de overheid wordt gehanteerd is beperkter. Dan gaat het om een directe relatie met het financiële beheer. In de LegitiMaat geven we daarom steeds aan welke vorm van rechtmatigheid we bedoelen.

Regelexpert medewerker die wetten vertaalt in regels.

Rekenregels regels waarmee een som gemaakt wordt met de variabelen

Repository een ICT/-architectenterm voor een bewaarplaats of magazijn.

Selectieregels met selectieregels kan worden geselecteerd welke zaken geautomatiseerd worden afgehandeld en welke door een medewerker moeten worden bekeken.

Softwareontwikkelaar een persoon die zich bezighoudt met het programmeren van software. Een softwareontwikkelaar wordt ook wel programmeur, computerprogrammeur of applicatieontwikkelaar genoemd.

Statistische algoritmen Een statistisch algoritme is een ML algoritme dat op basis van trainingsdata een voorspelling maakt voor nieuwe data. Deze voorspelling is een kansberekening en geeft de kans weer dat een bepaalde situatie zich voor zal doen. Dit kan gebaseerd zijn op data uit het verleden, maar ook op data uit vergelijkbare situaties.

Uitvoeringsbeleid Consistente gedragslijn die is neergelegd in interne werkinstructies en waarin wordt bepaald hoe wordt omgegaan met bepaalde gevallen en hoe de beoordelingsruimte wordt ingevuld.

Uitworp/uitval/niet glad geval bij geautomatiseerde uitvoering van wetten is het gebruikelijk dat bepaald wordt welke zaken volledig geautomatiseerd worden afgehandeld en welke zaken door een medewerker nader bekeken moeten worden. Dit bepalen wordt soms aan de hand van selectieregels gedaan.

Valideren iets geldig verklaren.

Verfiëren onderzoeken of iets juist is.

Vernietigd als een besluit bij de burger bekend is, kan de burger binnen 6 weken in bezwaar gaan. Krijgt de burger gelijk, dan wordt het besluit soms vernietigd: het bestaat juridisch dan niet meer. Ook de rechter kan het besluit vernietigen. Een besluit is dus in ieder geval tijdens zes weken niet definitief. Is er bezwaar gemaakt, dan staat het besluit niet in rechte vast totdat de beslissing op bezwaar is genomen. Of tot de rechter uitspraak heeft gedaan.

Voorbereidingshandelingen Op grond van de Awb leidt een aanvraag tot een besluit. Om tot dit besluit te komen worden voorbereidingshandelingen verricht. Een deel van het proces dat met De LegitiMaat wordt getoetst, valt samen met deze voorbereidingshandelingen.

9. Conformance

As well as sections marked as non-normative, all authoring guidelines, diagrams, examples, and notes in this specification are non-normative. Everything else in this specification is normative.

10. List of Figures

A. Index

A.1 Terms defined by this specification

A.2 Terms defined by reference