A lawyer’s guide to Large Language Models (LLMs)

LLMs are increasingly important for legal work. They power faster contract reviews, compliance checks, due diligence, and everyday drafting. On its own, an LLM is just a language model trained to predict the next word in a sequence. Tools like ChatGPT give direct access to these models, which is why you might see colleagues using them for quick answers.

But legal AI platforms go further. They take the raw capabilities of an LLM and add legal-specific training, workflows, security, and integrations. This makes them suitable for real legal work where confidentiality and accuracy matter.

To make the most of both, lawyers need to understand what an LLM is, how it works, what it cannot do, and how platforms built on top of them work.

What LLMs are, and what they’re not

An LLM is a type of artificial intelligence trained on massive amounts of text. Instead of “knowing” facts, it learns patterns in how words and sentences are used together. This lets it predict the next word in a sequence - similar to autocomplete on your phone, but on a far more advanced level.

Some LLMs are public and offered by companies such as OpenAI (ChatGPT) or Anthropic (Claude). Others can be run privately by a business or by a vendor that guarantees the data will not be shared outside.

It is important to understand that an LLM does not work like a database or a search engine. It does not retrieve stored facts. It generates responses based on patterns it has seen in training. This means it can sometimes produce answers that look convincing but are wrong. These mistakes are called “hallucinations.”

LLMs can also be adapted for specific fields. When trained or fine-tuned with examples from legal work, they become more accurate for legal tasks. Without that step, general-purpose models are usually not reliable enough for sensitive or high-risk legal matters.

Read more: Everything you need to know about agentic AI for legal work

How LLMs work under the hood

LLMs are powered by an attention mechanism. Unlike people, who read text line by line, an LLM looks at all the information in a prompt at once. This means that how you frame the prompt strongly affects the result.

The model generates text one word at a time. Each new word is chosen based on probabilities calculated from all the previous words. The same question may produce different answers because the model introduces small variations in this process.

Different models perform differently. For example, Claude is often seen as better for tone and style. OpenAI models are often stronger in structured reasoning.  

Practical prompting tips for lawyers

The quality of outputs depends on the quality of inputs. You can improve results by using these prompting practices:

Provide full context. State who you are, what the task is, and what the end goal should be. For example: “I am a commercial lawyer reviewing NDAs. Produce a first-pass redline highlighting confidentiality clauses that deviate from the standard template.”

Give precise instructions. Do not use vague requests such as “make this better English.” Instead, define what “better” means: plainer English, a more formal tone, or a shorter summary.

Use only positive examples. Show the structure or style you want. Avoid including “bad” examples, as the model still pays attention to them and may reproduce parts of them.

Set clear rules. For legal work, this can mean instructions such as “Always quote case law, never paraphrase” or “Cite legislation in full title and section number.”

Break down complex tasks. Rather than asking for an entire due diligence report in one step, begin with a plan, then build each section step by step. This produces more reliable and reviewable results.

Start fresh when refining. If repeated iterations drift away from your original request, restate the task in a new session with a clearer prompt.

Save and reuse prompts. Keep a standard system prompt or instruction block that you can paste into new sessions so the model consistently understands your role and objectives.

Experiment with format. The way you structure your instructions (bullet points, numbered lists, or headings) can affect how the model organises its response.

Think of it like giving instructions to a junior lawyer. The clearer and more structured the guidance, the stronger the result.

Common mistakes LLMs make

Anyone who has used ChatGPT or another model will have seen the quirks. Some common mistakes include:

Maths errors. LLMs are not calculators. They generate text based on patterns in historic data, which means they often fumble when asked to do sums or more complex calculations.

Hallucinations. Sometimes models will confidently invent facts, cases, or citations that look convincing but are not real.

Context drop-off. With long documents, some models only process the first section properly, missing clauses buried deeper inside.

Why do LLMs give different answers to the same question?

LLMs do not always give the same answer to the same question. This variation comes from how they generate text. The model produces responses one word at a time, choosing each new word based on probabilities. The highest-probability word is not always selected, which creates natural differences between outputs.

Most models also use a setting called temperature. At higher temperatures, the model explores less probable words and produces more varied responses. At lower temperatures, it tends to repeat the most likely phrasing, which makes it more predictable.

For example, if you ask an LLM to summarise a confidentiality clause, at a low temperature it might consistently produce:

“This clause prevents either party from disclosing confidential information to third parties.”

At a higher temperature, the same request might return alternative wordings, such as:

“Neither party may share confidential material with outsiders unless permitted by law.” or

“Confidential data must be kept private and not revealed to any external person.”

The meaning is similar, but the phrasing changes.

What LLMs can do for legal teams

LLMs excel at recognising and generating patterns in language. This makes them useful for legal work that revolves around documents, precedents, and structured text. When used well, they can help lawyers:

• Handle routine questions from business colleagues by drawing on policies or playbooks
• Speed up legal research by scanning and summarising large volumes of material
• Draft and redline contracts quickly, reducing hours of manual editing
• Produce clear and consistent memos, briefs, and email responses
• Run due diligence by extracting and condensing information from large document sets
• Flag unusual or risky clauses and compare drafts against standard templates
• Check contract terms against regulatory requirements such as GDPR, AML, ESG, or DORA
• Generate templates that adapt to contract type, jurisdiction, or party details

Types of LLM deployments lawyers should know about

Different LLM-based tools have different strengths. However, public models are not designed to handle sensitive information. They are best used for low-risk, general tasks. For anything involving sensitive data, secure legal AI platforms are the safer option.

Some tools give access to general-purpose LLMs through APIs. These include tools from providers like OpenAI or Anthropic. They are fast to deploy but raise concerns around privacy, data security, and legal risk.

Other tools are built on open-source models such as Mistral or LLaMA. These can be hosted in private cloud environments or on-premise, which gives more control over security and data handling. They often require more technical support.

A growing number of legal technology platforms are built on top of one or several LLMs. Being LLM-agnostic means a platform is not tied to a single model but can draw on whichever performs best for a given task. This matters because different models have different strengths. One might handle long contracts better, another may reason more effectively through complex regulations. It also avoids over-reliance on a single provider and keeps the platform adaptable as new models emerge. LEGALFLY is LLM-agnostic, so it can route tasks to the most suitable model while anonymising data to maintain confidentiality.

Read more: Confidence, reliability and validity at LEGALFLY