No entries match these filters. Clear filters
Giskard adds a modular v3 vulnerability scanner and RAG RAG Retrieval-augmented generation: looking up relevant documents and adding them to the prompt before the model answers. It grounds responses in private or current data without retraining. quality evaluator for agentic systems, replacing the v2 LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. scan and RAGET︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ GET THIS VERSION
copy
$ git clone --branch giskard-scan/v1.0.2 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout giskard-scan/v1.0.2
└──▷ USE IT
Red-team a customer support agent against OWASP OWASP A nonprofit foundation that publishes open standards, guides, and ranked vulnerability lists—most famously the OWASP Top 10—giving cyber tools a vendor-neutral benchmark to test or report against. LLM Top-10 threat categories including prompt injection prompt injection An attack where text the model reads, such as a web page, file or tool result, carries instructions that override the user's. It is the main security risk for agents with tools. and harmful content.
python
copy
import asyncio
from giskard.scan import vulnerability_scan
async def my_agent(inputs: str) -> str:
# Replace with your real agent call
return f'Echo: {inputs}'
async def main() -> None:
await vulnerability_scan(
target=my_agent,
description='A customer support chatbot for an e-commerce platform.',
languages=['en'],
)
asyncio.run(main())
Run automated red-teaming across OWASP LLM Top-10 threat categories — prompt injection, harmful content, stereotypes, and misinformation — with vulnerability_scan(target, description, languages) from giskard.scan.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Evaluate RAG knowledge-base quality with quality_scan and KnowledgeBase, the v3 successor to RAGET. Not checked yet Extend scans with custom adversarial generators by passing ScenarioGenerator instances to generate_suite, or registering them on vulnerability_suite_generator_registry. Not checked yet Install the scanner with pip install 'giskard[scan]' or pip install giskard-scan; LLM provider extras such as giskard[openai] or giskard[anthropic] are available separately. Not checked yet Opt out of telemetry by setting DO_NOT_TRACK=1 or GISKARD_TELEMETRY_DISABLED=1 as environment variables or keys in a .env file in the working directory. Not checked yet
5 details in Everything
└──▷ BREAKING ON UPGRADE
! Giskard v2 is no longer actively maintained; v3 (giskard-scan) replaces the v2 LLM scan and RAGET for agentic systems. The v2-only automatic tabular scan (giskard.Model + giskard.Dataset) and giskard.testing ML test suite are not planned for v3 — pin pip install 'giskard[llm]>2,<3' to keep them.
Giskard adds run_sync() convenience methods for checks and a validate_template function for agents︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ GET THIS VERSION
copy
$ git clone --branch giskard-core/v1.0.4 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout giskard-core/v1.0.4
└──▷ USE IT
Run a Giskard scenario in a plain script or CI pipeline without needing asyncio.run() or an existing event loop.
python
copy
from giskard.checks import Scenario, Groundedness
def get_answer(inputs: str) -> str:
return "Paris"
scenario = (
Scenario("test_france_capital")
.interact(inputs="What is the capital of France?", outputs=get_answer)
.check(
Groundedness(
name="answer is grounded",
context="France is in Western Europe. Its capital is Paris.",
)
)
)
result = scenario.run_sync()
result.print_report()
Call run_sync() directly on a scenario or suite to run evals from a plain script without managing an event loop.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Validate agent prompt templates before execution with the new validate_template function in giskard.agents. Not checked yet Checks that are unavailable on the Hub now warn and skip rather than failing the entire run. Not checked yet
Call run_sync() directly on a scenario or suite to run evals from a plain script without managing an event loop.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Validate agent prompt templates before execution with the new validate_template function in giskard.agents. Not checked yet
1 more in Everything
Giskard Core v1.0.3 adds an opt-in strict template policy for agents.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ GET THIS VERSION
copy
$ git clone --branch giskard-core/v1.0.3 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout giskard-core/v1.0.3
Giskard v3.0.1 adds SOM support for judges and lets set_default_generator accept plain model strings.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ GET THIS VERSION
copy
$ git clone --branch v3.0.1 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v3.0.1
└──▷ USE IT
Set the default generator for checks using a plain model string instead of a generator object.
python
copy
from giskard.checks import set_default_generator
set_default_generator('openai/gpt-4o-mini')
Allows set_default_generator to accept model strings directly, so callers can specify a generator by name without wrapping it in an object.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
Giskard core v1.0.2 adds SOM (Self-Organizing Map) support for LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. judges and a per-file lint/typecheck agent quality gate.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ GET THIS VERSION
copy
$ git clone --branch giskard-core/v1.0.2 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout giskard-core/v1.0.2
Adds agent quality rails — a per-file lint/typecheck gate plus uv guard — enforcing code quality checks on agent hook execution.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
Giskard checks now measure scan and hub export telemetry
└──▷ GET THIS VERSION
copy
$ git clone --branch giskard-checks/v1.0.4 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout giskard-checks/v1.0.4
Adds telemetry measurement for scan duration and Hub export operations via the telemetry module Not checked yet
1 detail in Everything
giskard-checks/v1.0.3
Aug 26, 2026
·
issue 057
giskard-checks v1.0.3 lets set_default_generator accept model name strings directly.
└──▷ GET THIS VERSION
copy
$ git clone --branch giskard-checks/v1.0.3 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout giskard-checks/v1.0.3
Giskard v3 rewrites as a modular monorepo with new LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. judge checks, OWASP OWASP A nonprofit foundation that publishes open standards, guides, and ranked vulnerability lists—most famously the OWASP Top 10—giving cyber tools a vendor-neutral benchmark to test or report against. red-teaming, Suite runner, and a lean LLM provider layer.
└──▷ GET THIS VERSION
copy
$ git clone --branch v3.0.0 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v3.0.0
└──▷ USE IT
Define and run a grounded-answer eval scenario with a built-in LLM judge, then print the report.
python
copy
import asyncio
from giskard.checks import Scenario, Groundedness
def get_answer(inputs: str) -> str:
return "Paris"
async def main() -> None:
scenario = (
Scenario("test_france_capital")
.interact(inputs="What is the capital of France?", outputs=get_answer)
.check(
Groundedness(
name="answer is grounded",
context="France is in Western Europe. Its capital is Paris.",
)
)
)
result = await scenario.run()
result.print_report()
asyncio.run(main())
Compose multiple checks with AllOf to require that agent output is both relevant and non-toxic.
python
copy
from giskard.checks import Scenario, AllOf, AnswerRelevance, Toxicity
scenario = (
Scenario("test_safe_response")
.interact(inputs="Tell me about climate change.", outputs=my_agent)
.check(
AllOf(
AnswerRelevance(name="relevant"),
Toxicity(name="non-toxic"),
)
)
)
Red-team an agent across OWASP LLM Top-10 categories including prompt injection prompt injection An attack where text the model reads, such as a web page, file or tool result, carries instructions that override the user's. It is the main security risk for agents with tools. and harmful content.
python
copy
import asyncio
from giskard.scan import vulnerability_scan
async def my_agent(inputs: str) -> str:
return f"Echo: {inputs}"
async def main() -> None:
await vulnerability_scan(
target=my_agent,
description="A customer support chatbot for an e-commerce platform.",
languages=["en"],
)
asyncio.run(main())
New Scenario API with .interact() and .check() chain lets you define multi-turn eval scenarios; Suite runs batches of scenarios with dynamic binding and supports Suite.append() chaining. Not checked yet Adds annotations support on scenario/trace objects, enabling structured metadata to be attached to evaluation runs. Not checked yet Adds AllOf, AnyOf, and Not check composition operators in giskard-checks for building compound assertions. Not checked yet Adds AnswerRelevance LLM-judge check to giskard-checks for evaluating whether agent responses are on-topic. Not checked yet Adds Toxicity LLM-judge check to giskard-checks for detecting harmful content in agent outputs. Not checked yet
+16 more
show less
Adds RegexMatching check to giskard-checks for pattern-based output validation, with regex timeout to mitigate ReDoS ReDoS A denial-of-service attack that exploits catastrophically backtracking regular expressions, causing a regex engine to consume exponential CPU time on crafted input; tools use detection of vulnerable patterns to flag risky code. . Not checked yet Adds JSONValidity check for asserting that agent outputs are well-formed JSON. Not checked yet Adds print_report() method to result models, along with scenario and check error details in suite reports. Not checked yet Adds minimal OWASP LLM suite generator covering LLM01 indirect injection to giskard-checks. Not checked yet Introduces giskard-scan Beta with vulnerability_scan (red-teaming across OWASP LLM Top-10 threat categories including prompt injection, harmful content, stereotypes, and misinformation) and quality_scan (RAG RAG Retrieval-augmented generation: looking up relevant documents and adding them to the prompt before the model answers. It grounds responses in private or current data without retraining. /knowledge-base quality evaluation). Not checked yet Introduces giskard-llm, a lean provider-agnostic LLM routing layer replacing litellm as the default LLM backend. Not checked yet Introduces personas and extended context support for UserSimulator in the generators module. Not checked yet Adds metadata parameter to the generator completion pipeline in giskard-agents. Not checked yet Adds step-level type discriminator and tool input coercion with output serialization to giskard-agents. Not checked yet Adds generator retry and timeout policies to giskard-agents and giskard-core. Not checked yet Adds generator-as-protocol-adapter pattern in giskard-agents, enabling generators to serve as protocol bridges. Not checked yet Adds optional PostHog analytics to giskard-core and checks; opt out before importing via DO_NOT_TRACK=1 or GISKARD_TELEMETRY_DISABLED=1 environment variables. Not checked yet Supports pydantic-compatible input types for checks. Not checked yet Supports running individual scenarios multiple times. Not checked yet Adds JUnit XML JUnit XML A widely adopted test-report file format, originally from the Java JUnit framework, that records test suite results as XML; cyber tools emit it so their output plugs into CI systems like Jenkins or GitLab without custom parsers. export for SuiteResult. Not checked yet Rewrites Giskard as a v3 monorepo with packages giskard-core, giskard-agents, and giskard-checks; installs via pip install giskard (checks), pip install 'giskard[scan]' (vulnerability/quality scan), or pip install 'giskard[openai]' (provider SDK); requires Python 3.12+. Not checked yet
21 details in Everything
└──▷ BREAKING ON UPGRADE
! Scenario.from_sequence is removed; step-based scenario construction via the .interact() chain is now required.! Jinja2 template parsing in Workflow.chat is now opt-in rather than the default, breaking any workflow that relied on automatic Jinja rendering of chat messages.! Templating support in conformity rules is dropped entirely.! The Interact injection is now name-based, changing how check dependencies are resolved inside scenarios.! The Interact API is replaced with a new API (refactor: new 'Interact' API); code using the previous Interact interface will break.! Giskard v2 is no longer actively maintained; the automatic tabular scan (giskard.Model + giskard.Dataset), giskard.testing ML test suite, and Giskard Hub are not present in v3.
giskard-core/v1.0.1rc1
Prerelease
Aug 18, 2026
·
issue 049
Giskard v1.0.1rc1 ships PEP 561 type stubs, tightened public types, and expanded prompt-injection prompt-injection An attack where text the model reads, such as a web page, file or tool result, carries instructions that override the user's. It is the main security risk for agents with tools. scenarios.
└──▷ GET THIS VERSION
copy
$ git clone --branch giskard-core/v1.0.1rc1 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout giskard-core/v1.0.1rc1
Ships py.typed marker files (PEP 561) for giskard-core, giskard-llm, giskard-checks, and giskard-scan, enabling static type checking in downstream projects. Not checked yet Exports and tightens public Literal /status types from giskard.types for stricter API contracts. Not checked yet Expands the prompt-injection test suite with additional scenarios and templates covering more injection attack patterns. Not checked yet
3 details in Everything
└──▷ BREAKING ON UPGRADE
! Persisted-scenario tree now rejects unknown fields — any saved scenario files containing unrecognized keys will fail to load after upgrading.! The value-under-test field is renamed to target_key on every check — any code or serialized data referencing the old field name will break.
giskard-core/v1.0.1b6
Prerelease
Aug 6, 2026
·
issue 037
Giskard v1.0.1b6 adds third-party scan integration and enforces non-blank judge reasons in checks.
└──▷ GET THIS VERSION
copy
$ git clone --branch giskard-core/v1.0.1b6 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout giskard-core/v1.0.1b6
Integrates third-party scan support into vulnerability_scan via the new feat(scan): integrate third party scan capability, extending the scanner beyond built-in generators. Not checked yet Enforces non-blank judge reasons in LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. -as-judge checks — evaluations that return an empty reason string now fail explicitly instead of silently passing. Not checked yet Exposes default registry generators on the public API (chore(scan): export default registry generators on the public API), making it easier to inspect and reuse built-in ScenarioGenerator instances. Not checked yet
3 details in Everything
giskard-scan/v1.0.0b3
Prerelease
Jul 13, 2026
·
issue 020
Giskard Scan adds LIDAR and DeepTeam third-party integrations for broader AI vulnerability coverage.
└──▷ GET THIS VERSION
copy
$ git clone --branch giskard-scan/v1.0.0b3 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout giskard-scan/v1.0.0b3
Integrates LIDAR scan as a third-party scan provider via third_party_scan. Not checked yet Integrates DeepTeam as a third-party scan provider for additional red-teaming coverage. Not checked yet
2 details in Everything
giskard-checks/v1.0.2b5
Prerelease
Jul 13, 2026
·
issue 020
Giskard Checks v1.0.2b5 adds LiDAR and DeepTeam third-party scan integrations to the vulnerability scanner.
└──▷ GET THIS VERSION
copy
$ git clone --branch giskard-checks/v1.0.2b5 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout giskard-checks/v1.0.2b5
Integrates LiDAR scan into the third-party scan pipeline via third_party_scan. Not checked yet Adds DeepTeam as a third-party scan integration for agent vulnerability scanning. Not checked yet
2 details in Everything
giskard-core/v1.0.1b5
Prerelease
Jul 13, 2026
·
issue 020
Giskard adds crescendo/GCG/do-not-answer/sycophancy scan generators, HarmBench dataset, vulnerability scan API, singleturn/multiturn mode, and new checks.
└──▷ GET THIS VERSION
copy
$ git clone --branch giskard-core/v1.0.1b5 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout giskard-core/v1.0.1b5
└──▷ USE IT
Run a full vulnerability scan against an agent with the new programmatic API, including multi-turn mode.
python
copy
import asyncio
from giskard.scan import vulnerability_scan
async def main():
await vulnerability_scan(
target=my_agent,
description="A customer support chatbot for an e-commerce platform.",
languages=["en"],
)
asyncio.run(main())
Export suite results to Hub-compatible format for upstream reporting or dashboards.
python
copy
result = await suite.run()
hub_payload = result.to_hub_format()
Adds crescendo-like multi-turn adversarial scenario generator for red-teaming LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. agents. Not checked yet Adds GCG (Greedy Coordinate Gradient) injection scenario generator for adversarial suffix attacks. Not checked yet Adds do-not-answer scan generator to probe refusal behavior. Not checked yet Adds sycophantic scenario generator derived from a knowledge base. Not checked yet Adds quality scan with knowledge base scenario generator for RAG RAG Retrieval-augmented generation: looking up relevant documents and adding them to the prompt before the model answers. It grounds responses in private or current data without retraining. and agent quality evaluation. Not checked yet
+13 more
show less
Adds automatic recommendations derived from scan failure categories. Not checked yet Integrates the HarmBench dataset into the vulnerability scan. Not checked yet Adds a vulnerability scan API (vulnerability_scan) for programmatic red-teaming. Not checked yet Adds target_mode flag to switch between single-turn and multi-turn scan execution. Not checked yet Adds readability NLP metric check to giskard-checks. Not checked yet Adds match mode to comparison checks for flexible string/value matching. Not checked yet Adds suite field to SuiteResult for richer result introspection. Not checked yet Adds to_hub_format() method on SuiteResult to export results to Hub-compatible format. Not checked yet Adds scenario tag grouping in suite results for organized reporting. Not checked yet Makes suite failure reporting configurable. Not checked yet Supports unlimited suite reporting (removes previous reporting cap). Not checked yet Adds pydantic-settings integration for default model and config management in checks. Not checked yet Adds fail-fast behavior when instantiating checks that require a missing optional library. Not checked yet
18 details in Everything
Giskard adds parallel Suite.run execution, GOAT scenario generator, Rego policy checks, stereotype/discrimination adversarial generation, and provider transport passthrough.
└──▷ GET THIS VERSION
copy
$ git clone --branch giskard-llm/v1.0.0b4 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout giskard-llm/v1.0.0b4
└──▷ USE IT
Run a validation suite in parallel to reduce overall evaluation time.
python
copy
suite.run(parallel=True)
Suite.run now accepts parallel=True to execute scenarios concurrently Not checked yet Adds GOAT scenario generator for vulnerability-focused testing Not checked yet Adds adversarial scenario generator targeting stereotypes and discrimination Not checked yet Enables passing provider transport configuration through Giskard's LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. integration Not checked yet
4 details in Everything
Giskard v1.0.0b3 adds OWASP OWASP A nonprofit foundation that publishes open standards, guides, and ranked vulnerability lists—most famously the OWASP Top 10—giving cyber tools a vendor-neutral benchmark to test or report against. LLM01 indirect injection checks and trace type inference via parameter annotations.
└──▷ GET THIS VERSION
copy
$ git clone --branch giskard-llm/v1.0.0b3 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout giskard-llm/v1.0.0b3
Adds minimal OWASP LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. suite generator covering LLM01 indirect injection Not checked yet Enables checks to infer trace type from the target second parameter annotation Not checked yet
2 details in Everything
Adds AllOf/AnyOf/Not composition operators, Toxicity and AnswerRelevance LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. judges, giskard-llm replacement, optional PostHog telemetry, GeoIP GeoIP A database that maps IP addresses to geographic locations; tools use it to attribute traffic or alerts to countries, cities, or ASNs without querying external services. disable option, and agent instructions
└──▷ GET THIS VERSION
copy
$ git clone --branch giskard-core/v1.0.1b3 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout giskard-core/v1.0.1b3
Adds AllOf, AnyOf, and Not check composition operators in giskard-checks to combine multiple validation rules Not checked yet Adds Toxicity LLM judge check for evaluating output toxicity via the checks framework Not checked yet Adds AnswerRelevance LLM judge check for assessing answer relevance against prompts Not checked yet Introduces agents instructions feature to define custom behavior for Giskard agents Not checked yet Adds optional PostHog analytics integration in giskard-core and checks via telemetry config Not checked yet
+3 more
show less
Enables disabling GeoIP tracking in analytics with a new configuration option Not checked yet Replaces LiteLLM with the new giskard-llm library as the lean LLM provider backend Not checked yet Allows scenarios to be executed multiple times without reinitialization Not checked yet
8 details in Everything
advisory CVE-2026-40319
Apr 13, 2026
·
issue 128
CVE-2026-40319: Regular Expression Denial of Service (ReDoS ReDoS A denial-of-service attack that exploits catastrophically backtracking regular expressions, causing a regex engine to consume exponential CPU time on crafted input; tools use detection of vulnerable patterns to flag risky code. ) in RegexMatching Check (CVSS CVSS Common Vulnerability Scoring System, a method for describing a vulnerability's characteristics and severity. Its score is not the probability that attackers will exploit it. 5.5, affects <= 1.0.1b1)︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
advisory CVE-2026-40320
Apr 13, 2026
·
issue 128
CVE-2026-40320: Unsandboxed Jinja2 Template Rendering in ConformityCheck (CVSS CVSS Common Vulnerability Scoring System, a method for describing a vulnerability's characteristics and severity. Its score is not the probability that attackers will exploit it. 7.8, affects <= 1.0.1b1)︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
Adds Junix XML export for SuiteResult in Giskard checks
└──▷ GET THIS VERSION
copy
$ git clone --branch giskard-checks/v1.0.2b1 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout giskard-checks/v1.0.2b1
└──▷ USE IT
Export a suite of check results to the new Junix XML format for integration with external systems.
python
copy
result.export(format="junix")
Junix is now supported as an export format for SuiteResult objects Not checked yet
1 detail in Everything
└──▷ BREAKING ON UPGRADE
! Templating in conformity rules is no longer supported
advisory CVE-2026-34172
Mar 26, 2026
·
issue 128
CVE-2026-34172: Server-side template injection via ChatWorkflow.chat() using non-sandboxed Jinja2 Environment (CVSS CVSS Common Vulnerability Scoring System, a method for describing a vulnerability's characteristics and severity. Its score is not the probability that attackers will exploit it. 8.8, affects <= 0.3.3)︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
Adds scenario and check error details to suite reports in giskard-checks
└──▷ GET THIS VERSION
copy
$ git clone --branch giskard-core/v1.0.1b1 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout giskard-core/v1.0.1b1
└──▷ BREAKING ON UPGRADE
! Removed Scenario.from_sequence method
giskard-checks v1.0.1a2 adds print_report to result models and makes Suite.append() chainable.
└──▷ GET THIS VERSION
copy
$ git clone --branch giskard-checks/v1.0.1a2 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout giskard-checks/v1.0.1a2
Adds print_report method to check result models for direct report printing Not checked yet Makes Suite.append() chainable by returning self instead of None Not checked yet
2 details in Everything
Adds metadata parameter to the generator completion pipeline and changes Interact injection to be name-based.
└──▷ GET THIS VERSION
copy
$ git clone --branch giskard-agents/v1.0.2a1 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout giskard-agents/v1.0.2a1
Changes Interact injection behavior from position-based to name-based matching Not checked yet
1 detail in Everything
└──▷ BREAKING ON UPGRADE
! Interact injection is now name-based, breaking previous positional matching assumptions
Giskard v1.0.1a1 introduces RegexMatching checks, scenario annotations, batch suite execution, and agent step/type coercion.
└──▷ GET THIS VERSION
copy
$ git clone --branch giskard-core/v1.0.1a1 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout giskard-core/v1.0.1a1
Add RegexMatching check implementation for LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. and prompt reliability evaluation Not checked yet Add support for annotations in Scenario and Trace objects Not checked yet Suite now supports running batches of scenarios with dynamic binding Not checked yet Introduce step-level type discriminator in agents (GAP-003) Not checked yet Add tool input coercion and output serialization capabilities to agents (GAP-005) Not checked yet
5 details in Everything
└──▷ BREAKING ON UPGRADE
! The project has been rewritten as a v3 monorepo with giskard-core, giskard-agents, and giskard-checks packages! Scenario factory scenario() has been removed in favor of a mutable class structure
Giskard v2.19.0 adds custom embeddings embeddings Lists of numbers a model produces to represent the meaning of text, images or code, so similar items sit close together. They power semantic search and RAG. support
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.19.0 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.19.0
Adds Groq model support to LLMClient
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.18.0 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.18.0
Giskard v2.17.0 adds QATestset push to Hugging Face Hub integration
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.17.0 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.17.0
└──▷ USE IT
Upload a generated QA testset to your HF Hub repository for sharing or downstream evaluation.
code
copy
testset.push_to_hub("username/my-qa-testset")
Adds numerical perturbation detector and RAGET banking supervision report support
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.16.1 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.16.1
Adds numerical_perturbation detector for identifying vulnerabilities in numerical inputs Not checked yet
1 detail in Everything
Giskard v2.16.0 adds LiteLLM support and a security policy
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.16.0 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.16.0
Adds LiteLLM provider support to allow routing model requests through LiteLLM gateways Not checked yet Adds a default security_policy mechanism for LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. evaluation workflows Not checked yet
2 details in Everything
advisory CVE-2024-52524
Nov 14, 2024
·
issue 128
CVE-2024-52524: ReDoS ReDoS A denial-of-service attack that exploits catastrophically backtracking regular expressions, causing a regex engine to consume exponential CPU time on crafted input; tools use detection of vulnerable patterns to flag risky code. in Giskard Scan text perturbation (affects <= 2.15.4)︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
Adds conversation_correctness detector for evaluating LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. conversation flows.
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.15.2 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.15.2
└──▷ USE IT
Evaluate the correctness of responses in an AI conversation chain using the new detector.
python
copy
from giskard import Dataset, Model
from giskard.detectors import conversation_correctness
dataset = Dataset(df, ...
model = Model(...)
detector = conversation_correctness()
results = detector.run(dataset, model)
New conversation_correctness detector Not checked yet
1 detail in Everything
Adds llm_base_url parameter to LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. client and Python 3.12 support in Giskard v2.15.0
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.15.0 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.15.0
Adds llm_base_url parameter to the llm.client.__init__ module for custom OpenAI-compatible endpoints Not checked yet Adds explicit compatibility for Python 3.12 Not checked yet
2 details in Everything
Giskard v2.14.5 adds packaged prompt injection prompt injection An attack where text the model reads, such as a web page, file or tool result, carries instructions that override the user's. It is the main security risk for agents with tools. datasets for testing LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. applications.
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.14.5 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.14.5
Adds max_issues_per_detector parameter to scanner configuration
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.14.3 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.14.3
Giskard v2.14.2 adds local suite saving and improves reliability with disabled Sentry and local data fallback.
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.14.2 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.14.2
save_to_file() method to Suite objects allows exporting test suites for offline or version-controlled storage Not checked yet Sentry error reporting is now disabled by default, reducing background network calls when the library is used in disconnected environments Not checked yet Data injection files are used locally as a fallback when remote downloads fail, improving robustness for offline testing workflows Not checked yet
3 details in Everything
Adds Gemini client integration and JSON report generation from scan results
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.14.1 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.14.1
└──▷ USE IT
Initialize the Gemini client to interact with Google's model for scanning or evaluation.
python
copy
from giskard import Client
client = Client()
gemini_client = client.llm("gemini")
Export your scan results to a JSON file for downstream processing or sharing.
python
copy
scan_results.to_json("my_scan_results.json")
Adds Gemini client support for generating model predictions and handling LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. interactions Not checked yet Enables to_json() method on scan results to export findings as a JSON file Not checked yet
2 details in Everything
Adds Llama Bedrock client support for AWS Bedrock inference
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.14.0 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.14.0
Llama model support via the aws-bedrock provider Not checked yet
1 detail in Everything
Giskard v2.13.0 adds NeMo Guardrails integration and a Statistical Parity Difference test.
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.13.0 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.13.0
Adds NeMoGuardrails scan to validate LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. guardrail compliance. Not checked yet Adds statistical_parity_difference metric to the statistical parity tests. Not checked yet
2 details in Everything
Giskard v2.12.0 adds native Bedrock embeddings embeddings Lists of numbers a model produces to represent the meaning of text, images or code, so similar items sit close together. They power semantic search and RAG. for Claude and Titan
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.12.0 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.12.0
└──▷ BREAKING ON UPGRADE
! The eval_prompt parameter has been removed from hallucination tests
Giskard v2.11.0 introduces new Raget visualization redesign and custom metrics support alongside minor fixes.
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.11.0 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.11.0
Adds Raget visualization with a redesigned UI layout for better analysis workflows Not checked yet Enables easier integration of custom metrics through improved API support in the detection engine Not checked yet
2 details in Everything
Adds support for custom LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. clients and new Brier score, monotonicity, and smoothness tests.
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.10.0 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.10.0
Add support for custom LLM clients to allow provider integration Not checked yet Add brier_score test for probability calibration evaluation Not checked yet Add monotonicity test for model behavior consistency checks Not checked yet Add smoothness test for output stability analysis Not checked yet
4 details in Everything
Giskard v2.9.0 adds multi-mlworker support, kernel auto-generation, and managed mlworker lifecycle methods
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.9.0 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.9.0
Adds ability to connect multiple external ML workers via create_project method parameters Not checked yet Introduces automatic kernel generation based on project dependencies Not checked yet Adds new methods to start and stop managed ML workers Not checked yet
3 details in Everything
Adds API for executing test suites and uploading results to the Hub
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.7.5 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.7.5
Integrates Sentry for error tracking Not checked yet
1 detail in Everything
Giskard v2.7.4 adds deterministic seeding to LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. generators and evaluators for reproducible test results.
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.7.4 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.7.4
Adds seed parameter to LLM generators and evaluators to ensure reproducibility Not checked yet
1 detail in Everything
Giskard v2.7.3 adds llm-as-a-judge per-row metadata requirements and fragment markdown report support.
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.7.3 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.7.3
llm-as-a-judge tests can now include per-row test metadata specifying requirements Not checked yet
1 detail in Everything
Adds RAG RAG Retrieval-augmented generation: looking up relevant documents and adding them to the prompt before the model answers. It grounds responses in private or current data without retraining. toolset for evaluating retrieval-augmented generation retrieval-augmented generation Retrieval-augmented generation: looking up relevant documents and adding them to the prompt before the model answers. It grounds responses in private or current data without retraining. pipelines
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.7.2 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.7.2
Introduces the RAGToolset class to evaluate Retrieval-Augmented Generation models Not checked yet
1 detail in Everything
Giskard v2.7.1 adds Number-to-Words transformation and LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. tool calls support
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.7.1 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.7.1
Added Number-to-Words Transformation as a new transform capability. Not checked yet Added tool calls support for LLM clients. Not checked yet Added scan Transformation Function to the Hub. Not checked yet
3 details in Everything
Giskard v2.7.0 adds LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. -as-judge ground truth tests and custom scanners
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.7.0 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.7.0
Introduces the ability to define a custom scanner via the CustomScanner class Not checked yet
1 detail in Everything
Adds to_junit() method and improved __str__ for Dataset/Model in v2.6.0
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.6.0 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.6.0
└──▷ USE IT
Export test results to JUnit XML JUnit XML A widely adopted test-report file format, originally from the Java JUnit framework, that records test suite results as XML; cyber tools emit it so their output plugs into CI systems like Jenkins or GitLab without custom parsers. for CI/CD integration or reporting tools.
code
copy
junit_xml = result.to_junit()
Adds a to_junit method to TestSuiteResult to export reports in JUnit XML format Not checked yet Updates Dataset.__str__ and Model.__str__ to display the object's name and ID instead of default representation Not checked yet
2 details in Everything
Giskard v2.5.3 adds a method to create Dataset from TestDetails and includes ggshield in pre-commit pre-commit An open-source framework for managing and running git hooks, maintained by the pre-commit project, that lets cyber tools enforce checks automatically on every commit before code reaches a repository. .
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.5.3 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.5.3
└──▷ USE IT
Convert existing test details into a usable Dataset object.
python
copy
dataset = giskard.Dataset.from_test_details(test_results)
print(dataset)
Adds Dataset.from_test_details() method to construct datasets directly from test results Not checked yet Adds ggshield to the project's pre-commit configuration for secret scanning secret scanning Checking files, code or other data for exposed credentials such as passwords, API keys and tokens. Findings help teams remove the secret and revoke or rotate it. Not checked yet
2 details in Everything
Giskard v2.5.2 automates imports for kwargs tests on Giskard Hub
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.5.2 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.5.2
└──▷ HOW TO FIND IT
When registering a new kwargs-based test in the Hub, skip manual dependency management to avoid import errors.
📍 # Add your kwargs test via the UI or SDK; imports are now handled automatically
Giskard v2.5.0 adds LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. control character handling and Suite.upload() modification support.
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.5.0 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.5.0
Adds Suite.upload() method to upload modifications to existing suite Not checked yet Add a flag to handle control characters inside LLM response decoding Not checked yet
2 details in Everything
Giskard v2.4.0 adds test result metadata, underconfidence metric fixes, OCR OCR Optical character recognition, converting text in an image or scanned document into machine-readable characters. It lets software search or process text that was stored as pixels. typo transformation, and LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. hub refactoring.
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.4.0 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.4.0
Adds BaseModel.meta fields exposed via @property methods for accessible test result metadata Not checked yet Adds new details to ground truth tests for richer test result information Not checked yet Adds OCR typo as a supported transformation type for synthetic data generation Not checked yet Refactors the internal structure of the LLM hub for better modularity and future extensibility Not checked yet
4 details in Everything
Giskard v2.3.0 allows cancelling Hub jobs and refactors ML worker code
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.3.0 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.3.0
Adds the ability to cancel Hub jobs via API or UI (JIRA GSK-1943) Not checked yet
1 detail in Everything
Giskard v2.2.0 adds speech-to-text typo transformation and refactors prompt injection prompt injection An attack where text the model reads, such as a web page, file or tool result, carries instructions that override the user's. It is the main security risk for agents with tools. detection.
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.2.0 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.2.0
Add speech-to-text typos transformation Not checked yet Complete refactoring of prompt injection Not checked yet
2 details in Everything
Adds LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. -as-judge per-row evaluation and data quality tests
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.1.3 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.1.3
Added llm_as_judge row evaluator for evaluating model outputs against an LLM criterion per row Not checked yet Added new data_quality tests to assess dataset integrity Not checked yet
2 details in Everything
Giskard v2.1.2 adds Fisher's exact test and permutation tests for slice metric significance, plus explicit cache persistence control.
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.1.2 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.1.2
Adds Fisher's exact test and permutation test support for slice metrics significance testing Not checked yet Adds explicit control over when to persist/read the internal cache via a new configuration option Not checked yet
2 details in Everything
Giskard v2.1.0 adds HuggingFace report templates and language support for LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. generators
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.1.0 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.1.0
Adds a HuggingFace report template for generating scan results. Not checked yet Adds language support to LLM generators to specify the output language. Not checked yet Adds new Python signature arguments (args, kwargs) to public model methods and all() . Not checked yet
3 details in Everything
Adds feature subset scanning, Azure LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. scan support, and metric field to LLM issues in v2.0.6
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.0.6 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.0.6
Enables running LLM scans using Azure OpenAI Not checked yet Includes a metric field in LLM issue objects Not checked yet
2 details in Everything
Adds AVID integration to Giskard for model auditing reports.
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.0.4 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.0.4
AVID integration Not checked yet
1 detail in Everything
Giskard v2.0.3 tracks LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. calls/tokens in scans and requires name/desc for text_generation
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.0.3 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.0.3
Adds scan tracking for the number of LLM calls and tokens used Not checked yet Makes name and description mandatory parameters in text_generation functions Not checked yet
2 details in Everything
Giskard v2.0 introduces correlation and spurious correlation detectors, MLflow integration, and WebSocket WebSocket A protocol that upgrades an HTTP connection into a persistent, full-duplex channel, letting a server push data to a client without repeated polling. Cyber tools use it for real-time event streaming and live command output. -based worker status updates.
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.0.0 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.0.0
CorrelationDetector available for detecting feature correlations in datasets Not checked yet SpuriousCorrelationDetector added to the test suite for identifying spurious correlations Not checked yet Integration with MLflow for experiment tracking and artifact logging Not checked yet Worker backend communication switched to WebSockets, enabling reactive worker status updates in the UI Not checked yet
4 details in Everything
Adds cost estimation capability to the LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. scan workflow in Giskard v2.0.0b34
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.0.0b34 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.0.0b34
Adds cost estimate data to LLM scan results for tracking inference expenses Not checked yet
1 detail in Everything
Giskard v2.0.0b33 adds LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. debug argument support and a cache worker pool
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.0.0b33 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.0.0b33
Passes the debug argument through to LLM tests to enable debugging output Not checked yet Introduces a cache worker pool to accelerate test execution Not checked yet
2 details in Everything
Giskard v2.0.0b30 adds metric_name to test results and fixes prompt wrapping.
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.0.0b30 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.0.0b30
Adds metric_name field to test result objects Not checked yet
1 detail in Everything
Adds LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. scan capability to Giskard for testing prompt injection prompt injection An attack where text the model reads, such as a web page, file or tool result, carries instructions that override the user's. It is the main security risk for agents with tools. and special character vulnerabilities
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.0.0b29 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.0.0b29
└──▷ USE IT
Use the Garak engine within an LLM scan to detect specific prompt injection vectors.
python
copy
from giskard import llm_scan
import garak
garak_config = {
'model_provider': 'garak',
'model_name': 'gpt-3.5-turbo'
}
scan_result = llm_scan(
model=my_model,
dataset=my_dataset,
scan_type='llm',
engine=garak_config
)
llm_scan() function available in the Python API for executing comprehensive LLM security scans. Not checked yet Supports Garak integration for detecting prompt injection vulnerabilities via the llm_scan interface. Not checked yet
2 details in Everything
Adds function_inputs to TestSuiteDTO, makes dataset/model suite inputs with auto-upload, and adds CTA transformation names
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.0.0b27 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.0.0b27
Adds function_inputs field to TestSuiteDTO for explicit input specification. Not checked yet Makes dataset and model inputs on the Suite class automatically uploaded when the suite is created. Not checked yet Displays the name of transformations performed in the CTA (Contextual Testing Agent) output. Not checked yet
3 details in Everything
Giskard v2.0.0b25 adds LangChain agent integration and debug_description parameter to test decorator
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.0.0b25 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.0.0b25
└──▷ USE IT
Add a custom description to a Giskard test for better debugging context
python
copy
@giskard.test(debug_description="Validates that prediction stability holds under noise")
def test_stability(df, model): ...
New debug_description field in the @giskard.test decorator for richer test context in debugging sessions Not checked yet
1 detail in Everything
└──▷ BREAKING ON UPGRADE
! The legacy ML worker info and properties have been removed
Adds LangChain agent integration for Giskard models and exposes client info (URL, tokens) in the display.
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.0.0b22 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.0.0b22
Adds langchain agent to interact with Giskard models Not checked yet Displays Giskard Client info including URL, token and Spaces token Not checked yet
2 details in Everything
Adds W&B integration, MLflow integration, and new LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. issue detectors.
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.0.0b16 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.0.0b16
[GSK-1504] Integration with W&B Not checked yet [GSK-1321] Integration with MLflow Not checked yet [GSK-1283] Additional LLM issue detectors Not checked yet [GSK-1473] Enabling rewrite_prompt for giskard evaluator Not checked yet Feature/persistent api keys Not checked yet
5 details in Everything
Adds websocket worker status updates and HuggingFace Spaces artifact upload to Giskard
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.0.0b12 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.0.0b12
Implements websocket support to get real-time worker status instead of polling Not checked yet Enables uploading test scan artifacts directly to a Hugging Face Spaces model repository Not checked yet Allows the scan widget to be optionally embedded in external applications Not checked yet
3 details in Everything
Giskard v2.0.0b9 adds correlation detector and project key copy support
└──▷ GET THIS VERSION
copy
$ git clone --branch v2.0.0b9 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v2.0.0b9
[GSK-1078] Correlation detector provides a new detection capability for feature correlations in models. Not checked yet [GSK-1297] Add ways to copy project key enables copying the project key directly from the UI. Not checked yet
2 details in Everything
Giskard v1.3.0 introduces External ML Worker for reusing Python environments, new Disparate Impact test, and arm64 Docker support.
└──▷ GET THIS VERSION
copy
$ git clone --branch v1.3.0 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v1.3.0
Adds External ML Worker capability to reuse existing Python code environments when executing models, reducing environment setup overhead Not checked yet Adds Disparate impact test as a new detection/test capability for fairness analysis Not checked yet Introduces arm64 architecture support via dedicated docker images Not checked yet
3 details in Everything
Giskard v1.2.0 adds custom filtering for regression tests, project key display, API token expiry date, and SHAP SHAP A Python library for explaining machine learning model predictions by assigning each input feature a contribution score, giving cyber tools interpretable output from otherwise opaque models. disabling.
└──▷ GET THIS VERSION
copy
$ git clone --branch v1.2.0 https://github.com/Giskard-AI/giskard-oss.git
$ git checkout v1.2.0
Enable Disable SHAP for a single feature to exclude specific features from SHAP explanations Not checked yet Add Custom filtering for regression capabilities to filter test results in regression mode Not checked yet Show API token expiry date in the UI to visualize when credentials expire Not checked yet Display project key on the projects page for easier identification and management Not checked yet
4 details in Everything