Giskard

No entries match these filters. Clear filters

1. giskard-core/v1.0.3 Oct 20, 2026 · issue 115

   Giskard Core v1.0.3 adds an opt-in strict template policy for agents.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

   └──▷ GET THIS VERSION

   copy

   $ git clone --branch giskard-core/v1.0.3 https://github.com/Giskard-AI/giskard-oss.git
   # already have the repo? check out this version:
   $ git checkout giskard-core/v1.0.3

2. v3.0.1 Oct 2, 2026 · issue 092

   Giskard v3.0.1 adds SOM support for judges and lets set_default_generator accept plain model strings.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

   └──▷ GET THIS VERSION

   copy

   $ git clone --branch v3.0.1 https://github.com/Giskard-AI/giskard-oss.git
   # already have the repo? check out this version:
   $ git checkout v3.0.1

   └──▷ USE IT
   Set the default generator for checks using a plain model string instead of a generator object.
   python copy

   from giskard.checks import set_default_generator
   
   set_default_generator('openai/gpt-4o-mini')

   • Allows set_default_generator to accept model strings directly, so callers can specify a generator by name without wrapping it in an object.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

   1 detail in Everything

3. giskard-core/v1.0.2 Oct 2, 2026 · issue 092

   Giskard core v1.0.2 adds SOM (Self-Organizing Map) support for LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. judges and a per-file lint/typecheck agent quality gate.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

   └──▷ GET THIS VERSION

   copy

   $ git clone --branch giskard-core/v1.0.2 https://github.com/Giskard-AI/giskard-oss.git
   # already have the repo? check out this version:
   $ git checkout giskard-core/v1.0.2

   • Adds agent quality rails — a per-file lint/typecheck gate plus uv guard — enforcing code quality checks on agent hook execution.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

   1 detail in Everything

4. giskard-checks/v1.0.4 Sep 14, 2026 · release notes

   Giskard checks now measure scan and hub export telemetry

   └──▷ GET THIS VERSION

   copy

   $ git clone --branch giskard-checks/v1.0.4 https://github.com/Giskard-AI/giskard-oss.git
   # already have the repo? check out this version:
   $ git checkout giskard-checks/v1.0.4

   • Adds telemetry measurement for scan duration and Hub export operations via the telemetry module

   1 detail in Everything

5. giskard-checks/v1.0.3 Aug 26, 2026 · issue 057

   giskard-checks v1.0.3 lets set_default_generator accept model name strings directly.

   └──▷ GET THIS VERSION

   copy

   $ git clone --branch giskard-checks/v1.0.3 https://github.com/Giskard-AI/giskard-oss.git
   # already have the repo? check out this version:
   $ git checkout giskard-checks/v1.0.3

6. v3.0.0 Aug 26, 2026 · issue 057

   Giskard v3 rewrites as a modular monorepo with new LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. judge checks, OWASPOWASPA nonprofit foundation that publishes open standards, guides, and ranked vulnerability lists—most famously the OWASP Top 10—giving cyber tools a vendor-neutral benchmark to test or report against. red-teaming, Suite runner, and a lean LLM provider layer.

   └──▷ GET THIS VERSION

   copy

   $ git clone --branch v3.0.0 https://github.com/Giskard-AI/giskard-oss.git
   # already have the repo? check out this version:
   $ git checkout v3.0.0

   └──▷ USE IT
   Define and run a grounded-answer eval scenario with a built-in LLM judge, then print the report.
   python copy

   import asyncio
   from giskard.checks import Scenario, Groundedness
   
   def get_answer(inputs: str) -> str:
       return "Paris"
   
   async def main() -> None:
       scenario = (
           Scenario("test_france_capital")
           .interact(inputs="What is the capital of France?", outputs=get_answer)
           .check(
               Groundedness(
                   name="answer is grounded",
                   context="France is in Western Europe. Its capital is Paris.",
               )
           )
       )
       result = await scenario.run()
       result.print_report()
   
   asyncio.run(main())

   Compose multiple checks with AllOf to require that agent output is both relevant and non-toxic.
   python copy

   from giskard.checks import Scenario, AllOf, AnswerRelevance, Toxicity
   
   scenario = (
       Scenario("test_safe_response")
       .interact(inputs="Tell me about climate change.", outputs=my_agent)
       .check(
           AllOf(
               AnswerRelevance(name="relevant"),
               Toxicity(name="non-toxic"),
           )
       )
   )

   Red-team an agent across OWASP LLM Top-10 categories including prompt injectionprompt injectionAn attack where text the model reads, such as a web page, file or tool result, carries instructions that override the user's. It is the main security risk for agents with tools. and harmful content.
   python copy

   import asyncio
   from giskard.scan import vulnerability_scan
   
   async def my_agent(inputs: str) -> str:
       return f"Echo: {inputs}"
   
   async def main() -> None:
       await vulnerability_scan(
           target=my_agent,
           description="A customer support chatbot for an e-commerce platform.",
           languages=["en"],
       )
   
   asyncio.run(main())

   • New Scenario API with .interact() and .check() chain lets you define multi-turn eval scenarios; Suite runs batches of scenarios with dynamic binding and supports Suite.append() chaining.
   • Adds annotations support on scenario/trace objects, enabling structured metadata to be attached to evaluation runs.
   • Adds AllOf, AnyOf, and Not check composition operators in giskard-checks for building compound assertions.
   • Adds AnswerRelevance LLM-judge check to giskard-checks for evaluating whether agent responses are on-topic.
   • Adds Toxicity LLM-judge check to giskard-checks for detecting harmful content in agent outputs.

   +16 more show less

   • Adds RegexMatching check to giskard-checks for pattern-based output validation, with regex timeout to mitigate ReDoSReDoSA denial-of-service attack that exploits catastrophically backtracking regular expressions, causing a regex engine to consume exponential CPU time on crafted input; tools use detection of vulnerable patterns to flag risky code..
   • Adds JSONValidity check for asserting that agent outputs are well-formed JSON.
   • Adds print_report() method to result models, along with scenario and check error details in suite reports.
   • Adds minimal OWASP LLM suite generator covering LLM01 indirect injection to giskard-checks.
   • Introduces giskard-scan (beta) with vulnerability_scan (red-teaming across OWASP LLM Top-10 threat categories including prompt injection, harmful content, stereotypes, and misinformation) and quality_scan (RAGRAGRetrieval-augmented generation: looking up relevant documents and adding them to the prompt before the model answers. It grounds responses in private or current data without retraining./knowledge-base quality evaluation).
   • Introduces giskard-llm, a lean provider-agnostic LLM routing layer replacing litellm as the default LLM backend.
   • Introduces personas and extended context support for UserSimulator in the generators module.
   • Adds metadata parameter to the generator completion pipeline in giskard-agents.
   • Adds step-level type discriminator and tool input coercion with output serialization to giskard-agents.
   • Adds generator retry and timeout policies to giskard-agents and giskard-core.
   • Adds generator-as-protocol-adapter pattern in giskard-agents, enabling generators to serve as protocol bridges.
   • Adds optional PostHog analytics to giskard-core and checks; opt out before importing via DO_NOT_TRACK=1 or GISKARD_TELEMETRY_DISABLED=1 environment variables.
   • Supports pydantic-compatible input types for checks.
   • Supports running individual scenarios multiple times.
   • Adds JUnit XMLJUnit XMLA widely adopted test-report file format, originally from the Java JUnit framework, that records test suite results as XML; cyber tools emit it so their output plugs into CI systems like Jenkins or GitLab without custom parsers. export for SuiteResult.
   • Rewrites Giskard as a v3 monorepo with packages giskard-core, giskard-agents, and giskard-checks; installs via pip install giskard (checks), pip install 'giskard[scan]' (vulnerability/quality scan), or pip install 'giskard[openai]' (provider SDK); requires Python 3.12+.

   21 details in Everything

   └──▷ BREAKING ON UPGRADE

   • !Scenario.from_sequence is removed; step-based scenario construction via the .interact() chain is now required.
   • !Jinja2 template parsing in Workflow.chat is now opt-in rather than the default, breaking any workflow that relied on automatic Jinja rendering of chat messages.
   • !Templating support in conformity rules is dropped entirely.
   • !The Interact injection is now name-based, changing how check dependencies are resolved inside scenarios.
   • !The Interact API is replaced with a new API (refactor: new 'Interact' API); code using the previous Interact interface will break.
   • !Giskard v2 is no longer actively maintained; the automatic tabular scan (giskard.Model + giskard.Dataset), giskard.testing ML test suite, and Giskard Hub are not present in v3.
7. giskard-core/v1.0.1rc1 Prerelease Aug 18, 2026 · issue 049

   Giskard v1.0.1rc1 ships PEP 561 type stubs, tightened public types, and expanded prompt-injectionprompt-injectionAn attack where text the model reads, such as a web page, file or tool result, carries instructions that override the user's. It is the main security risk for agents with tools. scenarios.

   └──▷ GET THIS VERSION

   copy

   $ git clone --branch giskard-core/v1.0.1rc1 https://github.com/Giskard-AI/giskard-oss.git
   # already have the repo? check out this version:
   $ git checkout giskard-core/v1.0.1rc1

   • Ships py.typed marker files (PEP 561) for giskard-core, giskard-llm, giskard-checks, and giskard-scan, enabling static type checking in downstream projects.
   • Exports and tightens public Literal/status types from giskard.types for stricter API contracts.
   • Expands the prompt-injection test suite with additional scenarios and templates covering more injection attack patterns.

   3 details in Everything

   └──▷ BREAKING ON UPGRADE

   • !Persisted-scenario tree now rejects unknown fields — any saved scenario files containing unrecognized keys will fail to load after upgrading.
   • !The value-under-test field is renamed to target_key on every check — any code or serialized data referencing the old field name will break.
8. giskard-core/v1.0.1b6 Prerelease Aug 6, 2026 · issue 037

   Giskard v1.0.1b6 adds third-party scan integration and enforces non-blank judge reasons in checks.

   └──▷ GET THIS VERSION

   copy

   $ git clone --branch giskard-core/v1.0.1b6 https://github.com/Giskard-AI/giskard-oss.git
   # already have the repo? check out this version:
   $ git checkout giskard-core/v1.0.1b6

   • Integrates third-party scan support into vulnerability_scan via the new feat(scan): integrate third party scan capability, extending the scanner beyond built-in generators.
   • Enforces non-blank judge reasons in LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code.-as-judge checks — evaluations that return an empty reason string now fail explicitly instead of silently passing.
   • Exposes default registry generators on the public API (chore(scan): export default registry generators on the public API), making it easier to inspect and reuse built-in ScenarioGenerator instances.

   3 details in Everything

9. giskard-scan/v1.0.0b3 Prerelease Jul 13, 2026 · issue 020

   Giskard Scan adds LIDAR and DeepTeam third-party integrations for broader AI vulnerability coverage.

   └──▷ GET THIS VERSION

   copy

   $ git clone --branch giskard-scan/v1.0.0b3 https://github.com/Giskard-AI/giskard-oss.git
   # already have the repo? check out this version:
   $ git checkout giskard-scan/v1.0.0b3

   • Integrates LIDAR scan as a third-party scan provider via third_party_scan.
   • Integrates DeepTeam as a third-party scan provider for additional red-teaming coverage.

   2 details in Everything

10. giskard-checks/v1.0.2b5 Prerelease Jul 13, 2026 · issue 020

    Giskard Checks v1.0.2b5 adds LiDAR and DeepTeam third-party scan integrations to the vulnerability scanner.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch giskard-checks/v1.0.2b5 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout giskard-checks/v1.0.2b5

    • Integrates LiDAR scan into the third-party scan pipeline via third_party_scan.
    • Adds DeepTeam as a third-party scan integration for agent vulnerability scanning.

    2 details in Everything

11. giskard-core/v1.0.1b5 Prerelease Jul 13, 2026 · issue 020

    Giskard adds crescendo/GCG/do-not-answer/sycophancy scan generators, HarmBench dataset, vulnerability scan API, singleturn/multiturn mode, and new checks.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch giskard-core/v1.0.1b5 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout giskard-core/v1.0.1b5

    └──▷ USE IT
    Run a full vulnerability scan against an agent with the new programmatic API, including multi-turn mode.
    python copy

    import asyncio
    from giskard.scan import vulnerability_scan
    
    async def main():
        await vulnerability_scan(
            target=my_agent,
            description="A customer support chatbot for an e-commerce platform.",
            languages=["en"],
        )
    
    asyncio.run(main())

    Export suite results to Hub-compatible format for upstream reporting or dashboards.
    python copy

    result = await suite.run()
    hub_payload = result.to_hub_format()

    • Adds crescendo-like multi-turn adversarial scenario generator for red-teaming LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. agents.
    • Adds GCG (Greedy Coordinate Gradient) injection scenario generator for adversarial suffix attacks.
    • Adds do-not-answer scan generator to probe refusal behavior.
    • Adds sycophantic scenario generator derived from a knowledge base.
    • Adds quality scan with knowledge base scenario generator for RAGRAGRetrieval-augmented generation: looking up relevant documents and adding them to the prompt before the model answers. It grounds responses in private or current data without retraining. and agent quality evaluation.

    +13 more show less

    • Adds automatic recommendations derived from scan failure categories.
    • Integrates the HarmBench dataset into the vulnerability scan.
    • Adds a vulnerability scan API (vulnerability_scan) for programmatic red-teaming.
    • Adds target_mode flag to switch between single-turn and multi-turn scan execution.
    • Adds readability NLP metric check to giskard-checks.
    • Adds match mode to comparison checks for flexible string/value matching.
    • Adds suite field to SuiteResult for richer result introspection.
    • Adds to_hub_format() method on SuiteResult to export results to Hub-compatible format.
    • Adds scenario tag grouping in suite results for organized reporting.
    • Makes suite failure reporting configurable.
    • Supports unlimited suite reporting (removes previous reporting cap).
    • Adds pydantic-settings integration for default model and config management in checks.
    • Adds fail-fast behavior when instantiating checks that require a missing optional library.

    18 details in Everything

12. giskard-llm/v1.0.0b4 Prerelease Jun 9, 2026 · release notes

    Giskard adds parallel Suite.run execution, GOAT scenario generator, Rego policy checks, stereotype/discrimination adversarial generation, and provider transport passthrough.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch giskard-llm/v1.0.0b4 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout giskard-llm/v1.0.0b4

    └──▷ USE IT
    Run a validation suite in parallel to reduce overall evaluation time.
    python copy

    suite.run(parallel=True)

    • Suite.run now accepts parallel=True to execute scenarios concurrently
    • Adds GOAT scenario generator for vulnerability-focused testing
    • Adds adversarial scenario generator targeting stereotypes and discrimination
    • Enables passing provider transport configuration through Giskard's LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. integration

    4 details in Everything

13. giskard-llm/v1.0.0b3 Prerelease May 20, 2026 · release notes

    Giskard v1.0.0b3 adds OWASPOWASPA nonprofit foundation that publishes open standards, guides, and ranked vulnerability lists—most famously the OWASP Top 10—giving cyber tools a vendor-neutral benchmark to test or report against. LLM01 indirect injection checks and trace type inference via parameter annotations.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch giskard-llm/v1.0.0b3 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout giskard-llm/v1.0.0b3

    • Adds minimal OWASP LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. suite generator covering LLM01 indirect injection
    • Enables checks to infer trace type from the target second parameter annotation

    2 details in Everything

14. giskard-core/v1.0.1b3 Prerelease Apr 29, 2026 · release notes

    Adds AllOf/AnyOf/Not composition operators, Toxicity and AnswerRelevance LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. judges, giskard-llm replacement, optional PostHog telemetry, GeoIPGeoIPA database that maps IP addresses to geographic locations; tools use it to attribute traffic or alerts to countries, cities, or ASNs without querying external services. disable option, and agent instructions

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch giskard-core/v1.0.1b3 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout giskard-core/v1.0.1b3

    • Adds AllOf, AnyOf, and Not check composition operators in giskard-checks to combine multiple validation rules
    • Adds Toxicity LLM judge check for evaluating output toxicity via the checks framework
    • Adds AnswerRelevance LLM judge check for assessing answer relevance against prompts
    • Introduces agents instructions feature to define custom behavior for Giskard agents
    • Adds optional PostHog analytics integration in giskard-core and checks via telemetry config

    +3 more show less

    • Enables disabling GeoIP tracking in analytics with a new configuration option
    • Replaces LiteLLM with the new giskard-llm library as the lean LLM provider backend
    • Allows scenarios to be executed multiple times without reinitialization

    8 details in Everything

15. advisory CVE-2026-40319 Apr 13, 2026 · issue 128
    • CVE-2026-40319: Regular Expression Denial of Service (ReDoSReDoSA denial-of-service attack that exploits catastrophically backtracking regular expressions, causing a regex engine to consume exponential CPU time on crafted input; tools use detection of vulnerable patterns to flag risky code.) in RegexMatching Check (CVSSCVSSCommon Vulnerability Scoring System, a method for describing a vulnerability's characteristics and severity. Its score is not the probability that attackers will exploit it. 5.5, affects <= 1.0.1b1)⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠
16. advisory CVE-2026-40320 Apr 13, 2026 · issue 128
    • CVE-2026-40320: Unsandboxed Jinja2 Template Rendering in ConformityCheck (CVSSCVSSCommon Vulnerability Scoring System, a method for describing a vulnerability's characteristics and severity. Its score is not the probability that attackers will exploit it. 7.8, affects <= 1.0.1b1)⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠
17. giskard-checks/v1.0.2b1 Prerelease Apr 10, 2026 · release notes

    Adds Junix XML export for SuiteResult in Giskard checks

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch giskard-checks/v1.0.2b1 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout giskard-checks/v1.0.2b1

    └──▷ USE IT
    Export a suite of check results to the new Junix XML format for integration with external systems.
    python copy

    result.export(format="junix")

    • Junix is now supported as an export format for SuiteResult objects

    1 detail in Everything

    └──▷ BREAKING ON UPGRADE

    • !Templating in conformity rules is no longer supported
18. advisory CVE-2026-34172 Mar 26, 2026 · issue 128
    • CVE-2026-34172: Server-side template injection via ChatWorkflow.chat() using non-sandboxed Jinja2 Environment (CVSSCVSSCommon Vulnerability Scoring System, a method for describing a vulnerability's characteristics and severity. Its score is not the probability that attackers will exploit it. 8.8, affects <= 0.3.3)⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠
19. giskard-core/v1.0.1b1 Prerelease Mar 26, 2026 · release notes

    Adds scenario and check error details to suite reports in giskard-checks

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch giskard-core/v1.0.1b1 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout giskard-core/v1.0.1b1

    └──▷ BREAKING ON UPGRADE

    • !Removed Scenario.from_sequence method
20. giskard-checks/v1.0.1a2 Prerelease Mar 20, 2026 · release notes

    giskard-checks v1.0.1a2 adds print_report to result models and makes Suite.append() chainable.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch giskard-checks/v1.0.1a2 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout giskard-checks/v1.0.1a2

    • Adds print_report method to check result models for direct report printing
    • Makes Suite.append() chainable by returning self instead of None

    2 details in Everything

21. giskard-agents/v1.0.2a1 Prerelease Mar 17, 2026 · release notes

    Adds metadata parameter to the generator completion pipeline and changes Interact injection to be name-based.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch giskard-agents/v1.0.2a1 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout giskard-agents/v1.0.2a1

    • Changes Interact injection behavior from position-based to name-based matching

    1 detail in Everything

    └──▷ BREAKING ON UPGRADE

    • !Interact injection is now name-based, breaking previous positional matching assumptions
22. giskard-core/v1.0.1a1 Prerelease Mar 12, 2026 · release notes

    Giskard v1.0.1a1 introduces RegexMatching checks, scenario annotations, batch suite execution, and agent step/type coercion.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch giskard-core/v1.0.1a1 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout giskard-core/v1.0.1a1

    • Add RegexMatching check implementation for LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. and prompt reliability evaluation
    • Add support for annotations in Scenario and Trace objects
    • Suite now supports running batches of scenarios with dynamic binding
    • Introduce step-level type discriminator in agents (GAP-003)
    • Add tool input coercion and output serialization capabilities to agents (GAP-005)

    5 details in Everything

    └──▷ BREAKING ON UPGRADE

    • !The project has been rewritten as a v3 monorepo with giskard-core, giskard-agents, and giskard-checks packages
    • !Scenario factory scenario() has been removed in favor of a mutable class structure
23. v2.19.0 Feb 10, 2026 · release notes

    Giskard v2.19.0 adds custom embeddingsembeddingsLists of numbers a model produces to represent the meaning of text, images or code, so similar items sit close together. They power semantic search and RAG. support

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.19.0 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.19.0

24. v2.18.0 Aug 18, 2025 · release notes

    Adds Groq model support to LLMClient

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.18.0 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.18.0

25. v2.17.0 Jun 11, 2025 · release notes

    Giskard v2.17.0 adds QATestset push to Hugging Face Hub integration

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.17.0 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.17.0

    └──▷ USE IT
    Upload a generated QA testset to your HF Hub repository for sharing or downstream evaluation.
    code copy

    testset.push_to_hub("username/my-qa-testset")

26. v2.16.1 Feb 12, 2025 · release notes

    Adds numerical perturbation detector and RAGET banking supervision report support

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.16.1 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.16.1

    • Adds numerical_perturbation detector for identifying vulnerabilities in numerical inputs

    1 detail in Everything

27. v2.16.0 Nov 21, 2024 · release notes

    Giskard v2.16.0 adds LiteLLM support and a security policy

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.16.0 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.16.0

    • Adds LiteLLM provider support to allow routing model requests through LiteLLM gateways
    • Adds a default security_policy mechanism for LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. evaluation workflows

    2 details in Everything

28. advisory CVE-2024-52524 Nov 14, 2024 · issue 128
    • CVE-2024-52524: ReDoSReDoSA denial-of-service attack that exploits catastrophically backtracking regular expressions, causing a regex engine to consume exponential CPU time on crafted input; tools use detection of vulnerable patterns to flag risky code. in Giskard Scan text perturbation (affects <= 2.15.4)⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠
29. v2.15.2 Oct 15, 2024 · release notes

    Adds conversation_correctness detector for evaluating LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. conversation flows.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.15.2 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.15.2

    └──▷ USE IT
    Evaluate the correctness of responses in an AI conversation chain using the new detector.
    python copy

    from giskard import Dataset, Model
    from giskard.detectors import conversation_correctness
    
    dataset = Dataset(df, ...
    model = Model(...)
    detector = conversation_correctness()
    results = detector.run(dataset, model)

    • New conversation_correctness detector

    1 detail in Everything

30. v2.15.0 Sep 3, 2024 · release notes

    Adds llm_base_url parameter to LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. client and Python 3.12 support in Giskard v2.15.0

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.15.0 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.15.0

    • Adds llm_base_url parameter to the llm.client.__init__ module for custom OpenAI-compatible endpoints
    • Adds explicit compatibility for Python 3.12

    2 details in Everything

31. v2.14.5 Aug 29, 2024 · release notes

    Giskard v2.14.5 adds packaged prompt injectionprompt injectionAn attack where text the model reads, such as a web page, file or tool result, carries instructions that override the user's. It is the main security risk for agents with tools. datasets for testing LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. applications.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.14.5 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.14.5

32. v2.14.3 Jul 25, 2024 · release notes

    Adds max_issues_per_detector parameter to scanner configuration

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.14.3 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.14.3

33. v2.14.2 Jul 17, 2024 · release notes

    Giskard v2.14.2 adds local suite saving and improves reliability with disabled Sentry and local data fallback.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.14.2 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.14.2

    • save_to_file() method to Suite objects allows exporting test suites for offline or version-controlled storage
    • Sentry error reporting is now disabled by default, reducing background network calls when the library is used in disconnected environments
    • Data injection files are used locally as a fallback when remote downloads fail, improving robustness for offline testing workflows

    3 details in Everything

34. v2.14.1 Jul 9, 2024 · release notes

    Adds Gemini client integration and JSON report generation from scan results

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.14.1 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.14.1

    └──▷ USE IT
    Initialize the Gemini client to interact with Google's model for scanning or evaluation.
    python copy

    from giskard import Client
    client = Client()
    gemini_client = client.llm("gemini")

    Export your scan results to a JSON file for downstream processing or sharing.
    python copy

    scan_results.to_json("my_scan_results.json")

    • Adds Gemini client support for generating model predictions and handling LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. interactions
    • Enables to_json() method on scan results to export findings as a JSON file

    2 details in Everything

35. v2.14.0 Jun 4, 2024 · release notes

    Adds Llama Bedrock client support for AWS Bedrock inference

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.14.0 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.14.0

    • Llama model support via the aws-bedrock provider

    1 detail in Everything

36. v2.13.0 May 28, 2024 · release notes

    Giskard v2.13.0 adds NeMo Guardrails integration and a Statistical Parity Difference test.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.13.0 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.13.0

    • Adds NeMoGuardrails scan to validate LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. guardrail compliance.
    • Adds statistical_parity_difference metric to the statistical parity tests.

    2 details in Everything

37. v2.12.0 May 3, 2024 · release notes

    Giskard v2.12.0 adds native Bedrock embeddingsembeddingsLists of numbers a model produces to represent the meaning of text, images or code, so similar items sit close together. They power semantic search and RAG. for Claude and Titan

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.12.0 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.12.0

    └──▷ BREAKING ON UPGRADE

    • !The eval_prompt parameter has been removed from hallucination tests
38. v2.11.0 Apr 19, 2024 · release notes

    Giskard v2.11.0 introduces new Raget visualization redesign and custom metrics support alongside minor fixes.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.11.0 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.11.0

    • Adds Raget visualization with a redesigned UI layout for better analysis workflows
    • Enables easier integration of custom metrics through improved API support in the detection engine

    2 details in Everything

39. v2.10.0 Apr 10, 2024 · release notes

    Adds support for custom LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. clients and new Brier score, monotonicity, and smoothness tests.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.10.0 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.10.0

    • Add support for custom LLM clients to allow provider integration
    • Add brier_score test for probability calibration evaluation
    • Add monotonicity test for model behavior consistency checks
    • Add smoothness test for output stability analysis

    4 details in Everything

40. v2.9.0 Apr 8, 2024 · release notes

    Giskard v2.9.0 adds multi-mlworker support, kernel auto-generation, and managed mlworker lifecycle methods

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.9.0 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.9.0

    • Adds ability to connect multiple external ML workers via create_project method parameters
    • Introduces automatic kernel generation based on project dependencies
    • Adds new methods to start and stop managed ML workers

    3 details in Everything

41. v2.7.5 Mar 6, 2024 · release notes

    Adds API for executing test suites and uploading results to the Hub

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.7.5 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.7.5

    • Integrates Sentry for error tracking

    1 detail in Everything

42. v2.7.4 Feb 28, 2024 · release notes

    Giskard v2.7.4 adds deterministic seeding to LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. generators and evaluators for reproducible test results.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.7.4 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.7.4

    • Adds seed parameter to LLM generators and evaluators to ensure reproducibility

    1 detail in Everything

43. v2.7.3 Feb 22, 2024 · release notes

    Giskard v2.7.3 adds llm-as-a-judge per-row metadata requirements and fragment markdown report support.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.7.3 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.7.3

    • llm-as-a-judge tests can now include per-row test metadata specifying requirements

    1 detail in Everything

44. v2.7.2 Feb 19, 2024 · release notes

    Adds RAGRAGRetrieval-augmented generation: looking up relevant documents and adding them to the prompt before the model answers. It grounds responses in private or current data without retraining. toolset for evaluating retrieval-augmented generationretrieval-augmented generationRetrieval-augmented generation: looking up relevant documents and adding them to the prompt before the model answers. It grounds responses in private or current data without retraining. pipelines

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.7.2 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.7.2

    • Introduces the RAGToolset class to evaluate Retrieval-Augmented Generation models

    1 detail in Everything

45. v2.7.1 Feb 13, 2024 · release notes

    Giskard v2.7.1 adds Number-to-Words transformation and LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. tool calls support

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.7.1 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.7.1

    • Added Number-to-Words Transformation as a new transform capability.
    • Added tool calls support for LLM clients.
    • Added scan Transformation Function to the Hub.

    3 details in Everything

46. v2.7.0 Jan 29, 2024 · release notes

    Giskard v2.7.0 adds LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code.-as-judge ground truth tests and custom scanners

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.7.0 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.7.0

    • Introduces the ability to define a custom scanner via the CustomScanner class

    1 detail in Everything

47. v2.6.0 Jan 23, 2024 · release notes

    Adds to_junit() method and improved __str__ for Dataset/Model in v2.6.0

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.6.0 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.6.0

    └──▷ USE IT
    Export test results to JUnit XMLJUnit XMLA widely adopted test-report file format, originally from the Java JUnit framework, that records test suite results as XML; cyber tools emit it so their output plugs into CI systems like Jenkins or GitLab without custom parsers. for CI/CD integration or reporting tools.
    code copy

    junit_xml = result.to_junit()

    • Adds a to_junit method to TestSuiteResult to export reports in JUnit XML format
    • Updates Dataset.__str__ and Model.__str__ to display the object's name and ID instead of default representation

    2 details in Everything

48. v2.5.3 Jan 19, 2024 · release notes

    Giskard v2.5.3 adds a method to create Dataset from TestDetails and includes ggshield in pre-commit.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.5.3 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.5.3

    └──▷ USE IT
    Convert existing test details into a usable Dataset object.
    python copy

    dataset = giskard.Dataset.from_test_details(test_results)
    print(dataset)

    • Adds Dataset.from_test_details() method to construct datasets directly from test results
    • Adds ggshield to the project's pre-commit configuration for secret scanningsecret scanningChecking files, code or other data for exposed credentials such as passwords, API keys and tokens. Findings help teams remove the secret and revoke or rotate it.

    2 details in Everything

49. v2.5.2 Jan 19, 2024 · release notes

    Giskard v2.5.2 automates imports for kwargs tests on Giskard Hub

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.5.2 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.5.2

    └──▷ HOW TO FIND IT
    When registering a new kwargs-based test in the Hub, skip manual dependency management to avoid import errors.
    📍# Add your kwargs test via the UI or SDK; imports are now handled automatically
50. v2.5.0 Jan 17, 2024 · release notes

    Giskard v2.5.0 adds LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. control character handling and Suite.upload() modification support.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.5.0 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.5.0

    • Adds Suite.upload() method to upload modifications to existing suite
    • Add a flag to handle control characters inside LLM response decoding

    2 details in Everything

51. v2.4.0 Jan 15, 2024 · release notes

    Giskard v2.4.0 adds test result metadata, underconfidence metric fixes, OCROCROptical character recognition, converting text in an image or scanned document into machine-readable characters. It lets software search or process text that was stored as pixels. typo transformation, and LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. hub refactoring.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.4.0 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.4.0

    • Adds BaseModel.meta fields exposed via @property methods for accessible test result metadata
    • Adds new details to ground truth tests for richer test result information
    • Adds OCR typo as a supported transformation type for synthetic data generation
    • Refactors the internal structure of the LLM hub for better modularity and future extensibility

    4 details in Everything

52. v2.3.0 Jan 4, 2024 · release notes

    Giskard v2.3.0 allows cancelling Hub jobs and refactors ML worker code

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.3.0 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.3.0

    • Adds the ability to cancel Hub jobs via API or UI (JIRA GSK-1943)

    1 detail in Everything

53. v2.2.0 Jan 3, 2024 · release notes

    Giskard v2.2.0 adds speech-to-text typo transformation and refactors prompt injectionprompt injectionAn attack where text the model reads, such as a web page, file or tool result, carries instructions that override the user's. It is the main security risk for agents with tools. detection.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.2.0 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.2.0

    • Add speech-to-text typos transformation
    • Complete refactoring of prompt injection

    2 details in Everything

54. v2.1.3 Dec 21, 2023 · release notes

    Adds LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code.-as-judge per-row evaluation and data quality tests

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.1.3 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.1.3

    • Added llm_as_judge row evaluator for evaluating model outputs against an LLM criterion per row
    • Added new data_quality tests to assess dataset integrity

    2 details in Everything

55. v2.1.2 Dec 18, 2023 · release notes

    Giskard v2.1.2 adds Fisher's exact test and permutation tests for slice metric significance, plus explicit cache persistence control.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.1.2 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.1.2

    • Adds Fisher's exact test and permutation test support for slice metrics significance testing
    • Adds explicit control over when to persist/read the internal cache via a new configuration option

    2 details in Everything

56. v2.1.0 Dec 11, 2023 · release notes

    Giskard v2.1.0 adds HuggingFace report templates and language support for LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. generators

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.1.0 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.1.0

    • Adds a HuggingFace report template for generating scan results.
    • Adds language support to LLM generators to specify the output language.
    • Adds new Python signature arguments (args, kwargs) to public model methods and all().

    3 details in Everything

57. v2.0.6 Dec 3, 2023 · release notes

    Adds feature subset scanning, Azure LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. scan support, and metric field to LLM issues in v2.0.6

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.0.6 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.0.6

    • Enables running LLM scans using Azure OpenAI
    • Includes a metric field in LLM issue objects

    2 details in Everything

58. v2.0.4 Nov 23, 2023 · release notes

    Adds AVID integration to Giskard for model auditing reports.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.0.4 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.0.4

    • AVID integration

    1 detail in Everything

59. v2.0.3 Nov 15, 2023 · release notes

    Giskard v2.0.3 tracks LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. calls/tokens in scans and requires name/desc for text_generation

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.0.3 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.0.3

    • Adds scan tracking for the number of LLM calls and tokens used
    • Makes name and description mandatory parameters in text_generation functions

    2 details in Everything

60. v2.0.0 Nov 6, 2023 · release notes

    Giskard v2.0 introduces correlation and spurious correlation detectors, MLflow integration, and WebSocketWebSocketA protocol that upgrades an HTTP connection into a persistent, full-duplex channel, letting a server push data to a client without repeated polling. Cyber tools use it for real-time event streaming and live command output.-based worker status updates.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.0.0 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.0.0

    • CorrelationDetector available for detecting feature correlations in datasets
    • SpuriousCorrelationDetector added to the test suite for identifying spurious correlations
    • Integration with MLflow for experiment tracking and artifact logging
    • Worker backend communication switched to WebSockets, enabling reactive worker status updates in the UI

    4 details in Everything

61. v2.0.0b34 Prerelease Nov 6, 2023 · release notes

    Adds cost estimation capability to the LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. scan workflow in Giskard v2.0.0b34

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.0.0b34 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.0.0b34

    • Adds cost estimate data to LLM scan results for tracking inference expenses

    1 detail in Everything

62. v2.0.0b33 Prerelease Nov 5, 2023 · release notes

    Giskard v2.0.0b33 adds LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. debug argument support and a cache worker pool

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.0.0b33 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.0.0b33

    • Passes the debug argument through to LLM tests to enable debugging output
    • Introduces a cache worker pool to accelerate test execution

    2 details in Everything

63. v2.0.0b30 Prerelease Oct 31, 2023 · release notes

    Giskard v2.0.0b30 adds metric_name to test results and fixes prompt wrapping.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.0.0b30 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.0.0b30

    • Adds metric_name field to test result objects

    1 detail in Everything

64. v2.0.0b29 Prerelease Oct 30, 2023 · release notes

    Adds LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. scan capability to Giskard for testing prompt injectionprompt injectionAn attack where text the model reads, such as a web page, file or tool result, carries instructions that override the user's. It is the main security risk for agents with tools. and special character vulnerabilities

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.0.0b29 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.0.0b29

    └──▷ USE IT
    Use the Garak engine within an LLM scan to detect specific prompt injection vectors.
    python copy

    from giskard import llm_scan
    import garak
    
    garak_config = {
        'model_provider': 'garak',
        'model_name': 'gpt-3.5-turbo'
    }
    
    scan_result = llm_scan(
        model=my_model,
        dataset=my_dataset,
        scan_type='llm',
        engine=garak_config
    )

    • llm_scan() function available in the Python API for executing comprehensive LLM security scans.
    • Supports Garak integration for detecting prompt injection vulnerabilities via the llm_scan interface.

    2 details in Everything

65. v2.0.0b27 Prerelease Oct 25, 2023 · release notes

    Adds function_inputs to TestSuiteDTO, makes dataset/model suite inputs with auto-upload, and adds CTA transformation names

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.0.0b27 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.0.0b27

    • Adds function_inputs field to TestSuiteDTO for explicit input specification.
    • Makes dataset and model inputs on the Suite class automatically uploaded when the suite is created.
    • Displays the name of transformations performed in the CTA (Contextual Testing Agent) output.

    3 details in Everything

66. v2.0.0b25 Prerelease Oct 12, 2023 · release notes

    Giskard v2.0.0b25 adds LangChain agent integration and debug_description parameter to test decorator

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.0.0b25 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.0.0b25

    └──▷ USE IT
    Add a custom description to a Giskard test for better debugging context
    python copy

    @giskard.test(debug_description="Validates that prediction stability holds under noise")
    def test_stability(df, model): ...

    • New debug_description field in the @giskard.test decorator for richer test context in debugging sessions

    1 detail in Everything

    └──▷ BREAKING ON UPGRADE

    • !The legacy ML worker info and properties have been removed
67. v2.0.0b22 Prerelease Sep 25, 2023 · release notes

    Adds LangChain agent integration for Giskard models and exposes client info (URL, tokens) in the display.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.0.0b22 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.0.0b22

    • Adds langchain agent to interact with Giskard models
    • Displays Giskard Client info including URL, token and Spaces token

    2 details in Everything

68. v2.0.0b16 Prerelease Sep 4, 2023 · release notes

    Adds W&B integration, MLflow integration, and new LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. issue detectors.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.0.0b16 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.0.0b16

    • [GSK-1504] Integration with W&B
    • [GSK-1321] Integration with MLflow
    • [GSK-1283] Additional LLM issue detectors
    • [GSK-1473] Enabling rewrite_prompt for giskard evaluator
    • Feature/persistent api keys

    5 details in Everything

69. v2.0.0b12 Prerelease Jul 25, 2023 · release notes

    Adds websocket worker status updates and HuggingFace Spaces artifact upload to Giskard

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.0.0b12 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.0.0b12

    • Implements websocket support to get real-time worker status instead of polling
    • Enables uploading test scan artifacts directly to a Hugging Face Spaces model repository
    • Allows the scan widget to be optionally embedded in external applications

    3 details in Everything

70. v2.0.0b9 Prerelease Jun 21, 2023 · release notes

    Giskard v2.0.0b9 adds correlation detector and project key copy support

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v2.0.0b9 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v2.0.0b9

    • [GSK-1078] Correlation detector provides a new detection capability for feature correlations in models.
    • [GSK-1297] Add ways to copy project key enables copying the project key directly from the UI.

    2 details in Everything

71. v1.3.0 Nov 23, 2022 · release notes

    Giskard v1.3.0 introduces External ML Worker for reusing Python environments, new Disparate Impact test, and arm64 Docker support.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v1.3.0 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v1.3.0

    • Adds External ML Worker capability to reuse existing Python code environments when executing models, reducing environment setup overhead
    • Adds Disparate impact test as a new detection/test capability for fairness analysis
    • Introduces arm64 architecture support via dedicated docker images

    3 details in Everything

72. v1.2.0 Sep 2, 2022 · release notes

    Giskard v1.2.0 adds custom filtering for regression tests, project key display, API token expiry date, and SHAPSHAPA Python library for explaining machine learning model predictions by assigning each input feature a contribution score, giving cyber tools interpretable output from otherwise opaque models. disabling.

    └──▷ GET THIS VERSION

    copy

    $ git clone --branch v1.2.0 https://github.com/Giskard-AI/giskard-oss.git
    # already have the repo? check out this version:
    $ git checkout v1.2.0

    • Enable Disable SHAP for a single feature to exclude specific features from SHAP explanations
    • Add Custom filtering for regression capabilities to filter test results in regression mode
    • Show API token expiry date in the UI to visualize when credentials expire
    • Display project key on the projects page for easier identification and management

    4 details in Everything