Choosing Between Deterministic Software, Decision Models and Large Language Models
LLMs make it easier to add AI to software, but AI is not always the right solution. This paper compares three approaches: deterministic software, decision/rule-based models, and LLM-based software, focusing on inputs, outputs, tokens, latency, context, security, and testability. It proposes a practical checklist for deciding when AI is appropriate, emphasizing problem characteristics, data, cost, latency, and long-term model dependency.
Abstract
The increasing availability of Large Language Models (LLMs) has made it easier to add AI capabilities to software. However, the availability of a technology does not necessarily mean that it is the appropriate solution for a problem.
This paper presents a simple framework for comparing three broad approaches: deterministic software, decision/rule-based models, and Large Language Model (LLM)-based software. The comparison focuses on the nature of inputs and outputs, token usage, latency, context limitations, security, and testability. It also proposes a practical checklist for deciding whether a problem is a suitable candidate for AI.
The central idea is that the choice of technology is related to how precisely the system's inputs, outputs, and expected behaviour can be specified. Deterministic software generally operates with explicitly specified behaviour. Decision systems can accept variable inputs while producing decisions within a defined space. LLM-based systems can operate on open-ended inputs and produce open-ended outputs, which creates additional challenges in evaluation, security, and system design. Research on software testing, machine learning production systems, and AI risk management supports the need to treat these systems differently rather than assuming that one testing or engineering approach applies equally to all of them.
1. Introduction
Software systems have traditionally been built by explicitly defining the behaviour that a program should perform. As machine learning and, more recently, LLMs have become widely available, another approach has emerged: instead of explicitly specifying every behaviour, part of the behaviour can be learned or generated by a model.
This creates an important engineering question:
When should a problem be solved using conventional software, a decision model, or an AI model?
The answer should not simply depend on whether AI can solve the problem. It should also consider the nature of the input, the nature of the output, how predictable the behaviour needs to be, how the system can be tested, latency requirements, security requirements, data availability, operating cost, and the dependency created by using an external model.
Research on software testing has long recognized that determining whether an observed output is correct can itself be difficult, known as the test oracle problem. However, machine learning systems introduce additional challenges because prediction behaviour can be difficult to specify in advance.
2. Three Types of Software
2.1 Deterministic Software
Deterministic software is based on explicitly defined logic. The important characteristic is not that the input itself is always fixed. A conventional application can receive millions of different inputs. Rather, the input domain and expected behaviour can be specified.
Conceptually:
Specified input domain → specified logic → specified behaviour
For example:
1
2
3
4
5
Input
↓
Business rules
↓
Expected result
The software engineer can define requirements, write code, create test cases and compare observed behaviour against expected behaviour.
The IEEE survey on the test oracle problem describes software testing as examining system behaviour to identify faults, while also noting that determining the correct behaviour for a given input can itself be a challenge.
Therefore, it would be inaccurate to say that deterministic software is “completely testable.” A more precise statement is that its behaviour is generally more explicitly specifiable, which allows extensive automated testing.
Typical applications: This approach is appropriate for software where rules and behaviour are known and can be explicitly implemented, such as conventional business logic and standard application processing. The important characteristic is not the industry but the predictability of the required behaviour.
2.2 Decision and Rule-Based Models
A decision model is different from simple deterministic processing because its input can be highly variable. The exact input is not known beforehand. However, the decision space can be defined.
Conceptually:
Variable input → decision rules/model → bounded decision
For example:
1
2
3
4
5
Variable input
↓
Decision model
↓
Decision A / Decision B
Rule-based decision systems commonly express policies using conditions and actions. IBM's documentation describes decision rules as conditions followed by actions and emphasizes completeness and consistency of decision logic.
NIST research on rule-based systems also describes approaches for pseudo-exhaustive testing of combinations of variable values on which a decision depends.
This leads to an important distinction:
Unknown or variable input does not necessarily mean unknown output.
A decision system can receive an input that was not explicitly known beforehand while still producing a result from a defined decision space.
This makes testing possible around decision boundaries, equivalence classes, edge cases, invalid inputs, boundary transitions and expected decisions.
Typical applications: Rule-based decision systems are used for business decision services and expert systems. NIST describes rule-based systems as important in business rule engines and other application domains. IBM's Decision Server documentation describes rule-based decision services that automate highly variable decisions requested by client applications.
2.3 Large Language Model-Based Software
LLM-based software has a different characteristic. Both the input and output can be open-ended.
Conceptually:
Open-ended input → LLM → open-ended output
For example, a user can ask the system a question in many different ways and receive a natural-language response that does not have to be one exact predefined string.
Recent research on LLM-based software engineering tools identifies three important evaluation difficulties: open-ended outputs, multiple valid answers, and non-deterministic behaviour across runs.
These characteristics challenge traditional assumptions of a single ground truth and deterministic output.
This does not mean that an LLM application cannot have constraints. An application can define allowed topics, output formats, validation rules, prohibited outputs, confidence or quality requirements, human review, tool permissions, and other application-level boundaries.
Therefore, the engineering problem changes from:
“What exact output should every input produce?”
towards:
“What behaviour is acceptable across the relevant input space?”
The ML Test Score research makes a similar observation for machine learning systems: prediction behaviour can be difficult to specify a priori, making testing and monitoring important parts of production readiness.
3. Comparison
| Dimension | Deterministic software | Decision / rule-based model | LLM-based software |
|---|---|---|---|
| Input | Specified input domain | Variable input | Potentially open-ended input |
| Output | Specified behaviour | Bounded decision space | Potentially open-ended output |
| Token dependency | No inherent token requirement | No inherent token requirement | Token-based model interfaces commonly affect cost and processing |
| Context window | Not an inherent concept | Not an inherent concept | Model context limits the amount of information available to the model |
| Latency | Usually determined by application logic and infrastructure | Depends on implementation and model/rule complexity | Depends on model, hardware, input/output size and serving architecture |
| Testability | Expected behaviour can usually be explicitly specified | Decision boundaries and rule combinations can be tested | Requires evaluation of acceptable behaviour, robustness and boundaries |
| Security | Conventional application security applies | Conventional application security plus decision integrity | Conventional security plus AI-specific risks |
| Main dependency | Code and infrastructure | Rules/model, data and infrastructure | Model, provider/infrastructure, prompts, data and evaluation |
| Behaviour | Explicitly programmed | Constrained by rules/model | Generated by the model within application constraints |
The table should not be interpreted as saying that every implementation has these exact characteristics. For example, a local LLM can have different latency and security properties from a hosted LLM. Similarly, a decision model may itself contain machine learning. The purpose is to compare the architectural characteristics of the three approaches.
3.1 Tokens
Tokens are an important operational consideration for LLM systems.
LLM inference can depend on the number of input and output tokens. AWS notes that token count affects computational requirements and inference cost, and that longer contexts can increase the cost of an inference request.
This creates an important difference from conventional software.
A conventional function such as:
1
calculate_tax(income)
does not have a token-based inference cost.
An LLM request may have:
1
2
3
4
5
input tokens
+
output tokens
+
model-specific processing
and therefore token usage becomes part of system economics and performance.
3.2 Latency
Latency is application-dependent rather than an inherent fixed property of AI.
LLM inference can introduce additional processing requirements, particularly as input and output token counts increase. AWS's production inference material explicitly treats latency, throughput and token-related computation as important deployment considerations.
Therefore, an AI solution should be evaluated against the actual latency requirement of the application.
A system that can tolerate seconds of processing is different from one that requires near-immediate responses.
3.3 Context Window
Context is a specific constraint of LLM systems.
An LLM can only process information within the context available to the model. When a conversation exceeds the input token limit, information may need to be truncated or otherwise managed.
This is fundamentally different from ordinary software variables or databases.
A database can contain a large amount of information without requiring all of it to be placed into a single model context.
Therefore:
Having access to data is not the same as having all of that data available to the model in one inference.
This distinction becomes particularly important in applications involving large documents, long conversations, retrieval systems or large codebases.
4. Security
Security is not unique to AI. Traditional software already has well-established security practices and vulnerabilities.
However, LLM applications introduce additional classes of security concerns.
OWASP's 2025 LLM and Generative AI Top 10 includes:
- prompt injection
- sensitive information disclosure
- supply-chain vulnerabilities
- data and model poisoning
- improper output handling
- excessive agency
- system prompt leakage
- vector and embedding weaknesses
- misinformation
- unbounded consumption
This means that an LLM application cannot be secured only by applying conventional API and application security practices.
The model and the way it interacts with users, data and external tools also need to be considered.
NIST similarly treats security, privacy, reliability, robustness and other characteristics as part of trustworthy AI evaluation.
This is particularly important when an LLM is connected to business systems or given the ability to take actions.
5. Testability
Testability is perhaps the largest difference between the three approaches.
Deterministic software
The engineer can generally define:
1
Input X → Expected Y
Testing can therefore compare actual and expected behaviour.
Decision models
Testing can focus on:
1
2
3
4
5
Input combinations
↓
Decision boundaries
↓
Expected decisions
NIST has demonstrated pseudo-exhaustive approaches for testing combinations of variable values in rule-based systems.
LLM systems
For many LLM applications, there may not be one exact correct output.
Testing therefore needs to consider:
- representative inputs
- edge cases
- robustness
- acceptable output properties
- security boundaries
- model behaviour
- monitoring after deployment
The ML Test Score paper identifies testing and monitoring as important for production ML systems because model prediction behaviour can be difficult to specify beforehand.
NIST also recommends ongoing testing and monitoring for deployed AI systems and emphasizes validity, reliability and robustness.
6. Proposed Checklist Before Adding AI
The following checklist is a practical framework proposed in this paper. It is not presented as a direct checklist taken from one existing reference.
1. Is the nature of the data sufficiently unpredictable?
Can conventional software reasonably represent the input and required behaviour through rules and explicitly defined logic?
If yes, adding AI may not provide enough additional value.
If the input is highly variable, ambiguous or difficult to describe through predefined rules, AI becomes a stronger candidate.
2. Do we have sufficient data?
Do we have enough representative data to develop and evaluate the system for the intended application?
Having a large quantity of data does not automatically mean that it is suitable for the problem.
Production ML systems have significant dependencies on data and require validation and monitoring around the data and model.
3. Is the data reliable and accessible?
Can the required data be obtained consistently?
Is it sufficiently reliable for the intended decision or task?
If the data cannot be trusted, the model cannot be expected to provide reliable system behaviour.
NIST's AI RMF emphasizes validity, reliability, robustness and evaluation as important characteristics of AI systems.
4. Is there sufficient volume of work to justify the cost?
AI introduces additional costs related to inference, infrastructure, model selection, evaluation, monitoring and maintenance.
For LLM systems, token usage can also directly affect inference cost.
Therefore, technical feasibility should be considered together with workload and economics.
5. Is inference latency acceptable?
Can the AI system respond within the latency requirements of the application?
The question is not simply:
“Can the model solve the problem?”
It is:
“Can it solve the problem within the required latency and operational constraints?”
LLM inference performance depends on factors including model architecture, workload, hardware and token generation.
6. How is the AI being used?
This is the most architectural question.
If the model is outsourced, does the model become the core business logic of the application?
Using an external model as a capability is different from making the model responsible for the application's fundamental business decisions.
ML research has identified risks including boundary erosion, entanglement, data dependencies and changes in the external world as sources of technical debt.
This suggests that AI should be given a clearly defined role rather than allowing model behaviour to silently become an uncontrolled business rule.
7. Trade-offs
There is no universally superior approach.
Deterministic software
Advantages
- Explicit behaviour
- Strong control over business logic
- Direct expected-output testing
- No model inference requirement
- No LLM context-window dependency
Trade-off
It may become difficult to maintain when the problem itself contains highly variable or unstructured inputs.
Decision / rule-based systems
Advantages
- Variable inputs can be handled
- Decision space can remain bounded
- Rules can be explicitly represented
- Decision logic can be tested for combinations and boundaries
Trade-off
The rule set can become large and complex. NIST notes that even simple expressions in rules can translate into substantial implementations, making systematic testing necessary.
LLM-based systems
Advantages
- Can work with open-ended natural-language input
- Can produce open-ended natural-language output
- Useful when exact rules are difficult to specify
- Can handle forms of information that are difficult to represent through fixed rules
Trade-offs
- Behaviour is harder to specify completely
- Evaluation becomes more complicated
- Token usage affects cost
- Context limits exist
- Latency must be evaluated
- Additional AI-specific security risks exist
- External model dependencies can introduce system-level risks
These trade-offs are consistent with research describing additional technical debt in ML systems and the challenges of evaluating open-ended LLM behaviour.
8. Applications
The appropriate technology depends on the structure of the problem.
Deterministic software
Suitable when:
- rules are explicitly known
- behaviour needs to be predictable
- exact business logic must be maintained
- the problem can be represented through conventional software
Decision and rule-based systems
Suitable when:
- inputs can vary considerably
- decisions can be expressed through rules
- the decision space is bounded
- consistency of the decision logic is important
Rule-based systems are used in business rule engines and decision services, where policies are expressed as rules and applied to variable inputs.
LLM-based systems
Suitable when:
- the input is open-ended or unstructured
- natural-language interpretation is required
- multiple valid outputs may exist
- the application can tolerate and manage model variability
LLMs therefore do not replace conventional software in general. They provide a different capability for problems where open-ended input and output are useful.
9. Conclusion
The decision to use AI should not begin with the question:
“Where can we add AI?”
It should begin with:
“What kind of problem are we solving?”
A useful conceptual distinction is:
Deterministic software
Specified input domain → specified behaviour
Decision model
Variable input → bounded decision
LLM
Open-ended input → open-ended output
As the input and output spaces become more open-ended, the ability to specify every possible correct behaviour becomes more difficult. Consequently, testing also changes—from direct expected-output testing toward decision-boundary testing, behavioural evaluation, robustness testing, security testing and continuous monitoring.
This does not make one approach universally better than another.
A deterministic system may be the appropriate engineering choice for one problem. A decision model may be appropriate for another. An LLM may be appropriate where the problem itself is difficult to describe using fixed rules.
The objective should therefore not be to maximize the amount of AI in a system.
It should be to choose the simplest architecture that can reliably solve the actual problem, while considering performance, security, cost, testability, data requirements and long-term system dependencies.
AI is an engineering option.
It is not automatically an engineering requirement.
References
- Barr, E. T., Harman, M., McMinn, P., Shahbaz, M., & Yoo, S. (2015). The Oracle Problem in Software Testing: A Survey. IEEE Transactions on Software Engineering, 41(5), 507–525.
- Breck, E., Cai, S., Nielsen, E., Salib, M., & Sculley, D. (2017). The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction. IEEE Big Data.
- Sculley, D., et al. (2015). Hidden Technical Debt in Machine Learning Systems. Advances in Neural Information Processing Systems 28.
- Kuhn, D. R., Yaga, D. J., Kacker, R. N., Lei, Y., & Hu, C. T. (2018). Pseudo-exhaustive Verification of Rule Based Systems. Proceedings of the 30th International Conference on Software Engineering and Knowledge Engineering.
- Tabassi, E. (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0). National Institute of Standards and Technology.
- NIST. AI Risk Management Framework — AI Risks and Trustworthiness. National Institute of Standards and Technology.
- OWASP GenAI Security Project. Top 10 Risks & Mitigations for LLMs and Generative AI Applications — 2025.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article