https://cybertoolchain.io/tools/giskard New releases and features in Giskard, tracked by The Cyber Toolchain. en Tue, 20 Oct 2026 11:24:34 GMT https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-core/v1.0.3 https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-core/v1.0.3 Tue, 20 Oct 2026 11:24:34 GMT Giskard Core v1.0.3 adds an opt-in strict template policy for agents.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠ • Adds an opt-in strict template policy to `giskard.agents`, controlling how prompt templates are enforced during agent runs.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠ Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v3.0.1 https://github.com/Giskard-AI/giskard-oss/releases/tag/v3.0.1 Fri, 02 Oct 2026 01:29:25 GMT Giskard v3.0.1 adds SOM support for judges and lets `set_default_generator` accept plain model strings.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠ • Allows `set_default_generator` to accept model strings directly, so callers can specify a generator by name without wrapping it in an object.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠ Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-core/v1.0.2 https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-core/v1.0.2 Fri, 02 Oct 2026 01:06:16 GMT Giskard core v1.0.2 adds SOM (Self-Organizing Map) support for LLM judges and a per-file lint/typecheck agent quality gate.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠ • Adds agent quality rails — a per-file lint/typecheck gate plus `uv` guard — enforcing code quality checks on agent hook execution.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠ Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-checks/v1.0.3 https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-checks/v1.0.3 Wed, 26 Aug 2026 08:39:15 GMT giskard-checks v1.0.3 lets `set_default_generator` accept model name strings directly. • Extends `set_default_generator` to accept model strings (e.g. a provider/model name) in addition to generator objects, simplifying default LLM generator configuration. Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v3.0.0 https://github.com/Giskard-AI/giskard-oss/releases/tag/v3.0.0 Wed, 26 Aug 2026 01:15:51 GMT Giskard v3 rewrites as a modular monorepo with new LLM judge checks, OWASP red-teaming, Suite runner, and a lean LLM provider layer. • New `Scenario` API with .interact() and .check() chain lets you define multi-turn eval scenarios; `Suite` runs batches of scenarios with dynamic binding and supports Suite.append() chaining. • Adds `annotations` support on scenario/trace objects, enabling structured metadata to be attached to evaluation runs. • Adds `AllOf`, `AnyOf`, and `Not` check composition operators in `giskard-checks` for building compound assertions. • Adds `AnswerRelevance` LLM-judge check to `giskard-checks` for evaluating whether agent responses are on-topic. • Adds `Toxicity` LLM-judge check to `giskard-checks` for detecting harmful content in agent outputs. • Adds `RegexMatching` check to `giskard-checks` for pattern-based output validation, with regex timeout to mitigate ReDoS. • Adds `JSONValidity` check for asserting that agent outputs are well-formed JSON. • Adds print_report() method to result models, along with scenario and check error details in suite reports. • Adds minimal OWASP LLM suite generator covering LLM01 indirect injection to `giskard-checks`. • Introduces `giskard-scan` (beta) with `vulnerability_scan` (red-teaming across OWASP LLM Top-10 threat categories including prompt injection, harmful content, stereotypes, and misinformation) and `quality_scan` (RAG/knowledge-base quality evaluation). • Introduces `giskard-llm`, a lean provider-agnostic LLM routing layer replacing litellm as the default LLM backend. • Introduces personas and extended context support for `UserSimulator` in the generators module. • Adds `metadata` parameter to the generator completion pipeline in `giskard-agents`. • Adds step-level type discriminator and tool input coercion with output serialization to `giskard-agents`. • Adds generator retry and timeout policies to `giskard-agents` and `giskard-core`. • Adds generator-as-protocol-adapter pattern in `giskard-agents`, enabling generators to serve as protocol bridges. • Adds optional PostHog analytics to `giskard-core` and checks; opt out before importing via `DO_NOT_TRACK=1` or `GISKARD_TELEMETRY_DISABLED=1` environment variables. • Supports pydantic-compatible input types for checks. • Supports running individual scenarios multiple times. • Adds JUnit XML export for `SuiteResult`. • Rewrites Giskard as a v3 monorepo with packages `giskard-core`, `giskard-agents`, and `giskard-checks`; installs via `pip install giskard` (checks), `pip install 'giskard[scan]'` (vulnerability/quality scan), or `pip install 'giskard[openai]'` (provider SDK); requires Python 3.12+. Breaking changes: • `Scenario.from_sequence` is removed; step-based scenario construction via the .interact() chain is now required. • Jinja2 template parsing in `Workflow.chat` is now opt-in rather than the default, breaking any workflow that relied on automatic Jinja rendering of chat messages. • Templating support in conformity rules is dropped entirely. • The `Interact` injection is now name-based, changing how check dependencies are resolved inside scenarios. • The 'Interact' API is replaced with a new API (`refactor: new 'Interact' API`); code using the previous Interact interface will break. • Giskard v2 is no longer actively maintained; the automatic tabular scan (`giskard.Model` + `giskard.Dataset`), `giskard.testing` ML test suite, and Giskard Hub are not present in v3. Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-core/v1.0.1rc1 https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-core/v1.0.1rc1 Tue, 18 Aug 2026 03:16:50 GMT Giskard v1.0.1rc1 ships PEP 561 type stubs, tightened public types, and expanded prompt-injection scenarios. • Ships `py.typed` marker files (PEP 561) for `giskard-core`, `giskard-llm`, `giskard-checks`, and `giskard-scan`, enabling static type checking in downstream projects. • Exports and tightens public `Literal`/status types from `giskard.types` for stricter API contracts. • Expands the prompt-injection test suite with additional scenarios and templates covering more injection attack patterns. Breaking changes: • Persisted-scenario tree now rejects unknown fields — any saved scenario files containing unrecognized keys will fail to load after upgrading. • The value-under-test field is renamed to `target_key` on every check — any code or serialized data referencing the old field name will break. Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-core/v1.0.1b6 https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-core/v1.0.1b6 Thu, 06 Aug 2026 03:53:52 GMT Giskard v1.0.1b6 adds third-party scan integration and enforces non-blank judge reasons in checks. • Integrates third-party scan support into `vulnerability_scan` via the new `feat(scan): integrate third party scan` capability, extending the scanner beyond built-in generators. • Enforces non-blank judge reasons in LLM-as-judge checks — evaluations that return an empty reason string now fail explicitly instead of silently passing. • Exposes default registry generators on the public API (`chore(scan): export default registry generators on the public API`), making it easier to inspect and reuse built-in `ScenarioGenerator` instances. Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-scan/v1.0.0b3 https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-scan/v1.0.0b3 Mon, 13 Jul 2026 08:49:31 GMT Giskard Scan adds LIDAR and DeepTeam third-party integrations for broader AI vulnerability coverage. • Integrates LIDAR scan as a third-party scan provider via `third_party_scan`. • Integrates DeepTeam as a third-party scan provider for additional red-teaming coverage. Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-checks/v1.0.2b5 https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-checks/v1.0.2b5 Mon, 13 Jul 2026 08:46:21 GMT Giskard Checks v1.0.2b5 adds LiDAR and DeepTeam third-party scan integrations to the vulnerability scanner. • Integrates LiDAR scan into the third-party scan pipeline via `third_party_scan`. • Adds DeepTeam as a third-party scan integration for agent vulnerability scanning. Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-core/v1.0.1b5 https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-core/v1.0.1b5 Mon, 13 Jul 2026 08:24:16 GMT Giskard adds crescendo/GCG/do-not-answer/sycophancy scan generators, HarmBench dataset, vulnerability scan API, singleturn/multiturn mode, and new checks. • Adds crescendo-like multi-turn adversarial scenario generator for red-teaming LLM agents. • Adds GCG (Greedy Coordinate Gradient) injection scenario generator for adversarial suffix attacks. • Adds do-not-answer scan generator to probe refusal behavior. • Adds sycophantic scenario generator derived from a knowledge base. • Adds quality scan with knowledge base scenario generator for RAG and agent quality evaluation. • Adds automatic recommendations derived from scan failure categories. • Integrates the HarmBench dataset into the vulnerability scan. • Adds a vulnerability scan API (`vulnerability_scan`) for programmatic red-teaming. • Adds `target_mode` flag to switch between single-turn and multi-turn scan execution. • Adds readability NLP metric check to `giskard-checks`. • Adds match mode to comparison checks for flexible string/value matching. • Adds `suite` field to `SuiteResult` for richer result introspection. • Adds to_hub_format() method on `SuiteResult` to export results to Hub-compatible format. • Adds scenario tag grouping in suite results for organized reporting. • Makes suite failure reporting configurable. • Supports unlimited suite reporting (removes previous reporting cap). • Adds pydantic-settings integration for default model and config management in checks. • Adds fail-fast behavior when instantiating checks that require a missing optional library. Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-llm/v1.0.0b4 https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-llm/v1.0.0b4 Tue, 09 Jun 2026 08:11:57 GMT Giskard adds parallel Suite.run execution, GOAT scenario generator, Rego policy checks, stereotype/discrimination adversarial generation, and provider transport passthrough. • `Suite.run` now accepts `parallel=True` to execute scenarios concurrently • Adds `GOAT` scenario generator for vulnerability-focused testing • Adds adversarial scenario generator targeting stereotypes and discrimination • Enables passing provider transport configuration through Giskard's LLM integration Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-llm/v1.0.0b3 https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-llm/v1.0.0b3 Wed, 20 May 2026 09:59:50 GMT Giskard v1.0.0b3 adds OWASP LLM01 indirect injection checks and trace type inference via parameter annotations. • Adds minimal OWASP LLM suite generator covering LLM01 indirect injection • Enables `checks` to infer trace type from the target second parameter annotation Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-core/v1.0.1b3 https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-core/v1.0.1b3 Wed, 29 Apr 2026 01:58:01 GMT Adds AllOf/AnyOf/Not composition operators, Toxicity and AnswerRelevance LLM judges, giskard-llm replacement, optional PostHog telemetry, GeoIP disable option, and agent instructions • Adds `AllOf`, `AnyOf`, and `Not` check composition operators in `giskard-checks` to combine multiple validation rules • Adds `Toxicity` LLM judge check for evaluating output toxicity via the checks framework • Adds `AnswerRelevance` LLM judge check for assessing answer relevance against prompts • Introduces `agents instructions` feature to define custom behavior for Giskard agents • Adds optional PostHog analytics integration in `giskard-core` and `checks` via telemetry config • Enables disabling GeoIP tracking in analytics with a new configuration option • Replaces LiteLLM with the new `giskard-llm` library as the lean LLM provider backend • Allows scenarios to be executed multiple times without reinitialization Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-checks/v1.0.2b1 https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-checks/v1.0.2b1 Fri, 10 Apr 2026 09:21:20 GMT Adds Junix XML export for SuiteResult in Giskard checks • `Junix` is now supported as an export format for `SuiteResult` objects Breaking changes: • Templating in conformity rules is no longer supported Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-core/v1.0.1b1 https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-core/v1.0.1b1 Thu, 26 Mar 2026 09:52:38 GMT Adds scenario and check error details to suite reports in giskard-checks • Shows scenario and check error details in suite report Breaking changes: • Removed `Scenario.from_sequence` method Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-checks/v1.0.1a2 https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-checks/v1.0.1a2 Fri, 20 Mar 2026 02:14:31 GMT giskard-checks v1.0.1a2 adds print_report to result models and makes Suite.append() chainable. • Adds `print_report` method to check result models for direct report printing • Makes Suite.append() chainable by returning self instead of None Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-agents/v1.0.2a1 https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-agents/v1.0.2a1 Tue, 17 Mar 2026 11:07:51 GMT Adds `metadata` parameter to the generator completion pipeline and changes Interact injection to be name-based. • Changes Interact injection behavior from position-based to name-based matching Breaking changes: • Interact injection is now name-based, breaking previous positional matching assumptions Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-core/v1.0.1a1 https://github.com/Giskard-AI/giskard-oss/releases/tag/giskard-core/v1.0.1a1 Thu, 12 Mar 2026 08:26:28 GMT Giskard v1.0.1a1 introduces RegexMatching checks, scenario annotations, batch suite execution, and agent step/type coercion. • Add `RegexMatching` check implementation for LLM and prompt reliability evaluation • Add support for `annotations` in `Scenario` and `Trace` objects • `Suite` now supports running batches of scenarios with dynamic binding • Introduce step-level type discriminator in agents (GAP-003) • Add tool input coercion and output serialization capabilities to agents (GAP-005) Breaking changes: • The project has been rewritten as a v3 monorepo with `giskard-core`, `giskard-agents`, and `giskard-checks` packages • Scenario factory scenario() has been removed in favor of a mutable class structure Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.19.0 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.19.0 Tue, 10 Feb 2026 10:14:17 GMT Giskard v2.19.0 adds custom embeddings support • Adds support for custom embeddings Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.18.0 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.18.0 Mon, 18 Aug 2025 08:28:08 GMT Adds Groq model support to LLMClient • Adds Groq model support to `LLMClient` Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.17.0 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.17.0 Wed, 11 Jun 2025 11:55:23 GMT Giskard v2.17.0 adds QATestset push to Hugging Face Hub integration • Adds QATestset.push_to_hub() to upload quantization-aware testsets directly to Hugging Face Hub Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.16.1 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.16.1 Wed, 12 Feb 2025 08:12:05 GMT Adds numerical perturbation detector and RAGET banking supervision report support • Adds `numerical_perturbation` detector for identifying vulnerabilities in numerical inputs Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.16.0 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.16.0 Thu, 21 Nov 2024 22:54:44 GMT Giskard v2.16.0 adds LiteLLM support and a security policy • Adds `LiteLLM` provider support to allow routing model requests through LiteLLM gateways • Adds a default `security_policy` mechanism for LLM evaluation workflows Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.15.2 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.15.2 Tue, 15 Oct 2024 12:20:05 GMT Adds `conversation_correctness` detector for evaluating LLM conversation flows. • New `conversation_correctness` detector Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.15.0 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.15.0 Tue, 03 Sep 2024 10:30:18 GMT Adds `llm_base_url` parameter to LLM client and Python 3.12 support in Giskard v2.15.0 • Adds `llm_base_url` parameter to the `llm.client.__init__` module for custom OpenAI-compatible endpoints • Adds explicit compatibility for Python 3.12 Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.14.5 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.14.5 Thu, 29 Aug 2024 08:47:42 GMT Giskard v2.14.5 adds packaged prompt injection datasets for testing LLM applications. • Adds prompt injection datasets to the library's packaged data for immediate use in scanning. Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.14.3 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.14.3 Thu, 25 Jul 2024 13:17:29 GMT Adds `max_issues_per_detector` parameter to scanner configuration • `max_issues_per_detector` is added as a parameter inside the scanner. Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.14.2 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.14.2 Wed, 17 Jul 2024 12:34:53 GMT Giskard v2.14.2 adds local suite saving and improves reliability with disabled Sentry and local data fallback. • save_to_file() method to `Suite` objects allows exporting test suites for offline or version-controlled storage • Sentry error reporting is now disabled by default, reducing background network calls when the library is used in disconnected environments • Data injection files are used locally as a fallback when remote downloads fail, improving robustness for offline testing workflows Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.14.1 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.14.1 Tue, 09 Jul 2024 05:51:31 GMT Adds Gemini client integration and JSON report generation from scan results • Adds `Gemini` client support for generating model predictions and handling LLM interactions • Enables to_json() method on scan results to export findings as a JSON file Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.14.0 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.14.0 Tue, 04 Jun 2024 15:13:07 GMT Adds Llama Bedrock client support for AWS Bedrock inference • `Llama` model support via the `aws-bedrock` provider Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.13.0 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.13.0 Tue, 28 May 2024 15:39:22 GMT Giskard v2.13.0 adds NeMo Guardrails integration and a Statistical Parity Difference test. • Adds `NeMoGuardrails` scan to validate LLM guardrail compliance. • Adds `statistical_parity_difference` metric to the statistical parity tests. Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.11.0 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.11.0 Fri, 19 Apr 2024 15:53:10 GMT Giskard v2.11.0 introduces new Raget visualization redesign and custom metrics support alongside minor fixes. • Adds `Raget` visualization with a redesigned UI layout for better analysis workflows • Enables easier integration of custom metrics through improved API support in the detection engine Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.10.0 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.10.0 Wed, 10 Apr 2024 17:18:26 GMT Adds support for custom LLM clients and new Brier score, monotonicity, and smoothness tests. • Add support for custom LLM clients to allow provider integration • Add `brier_score` test for probability calibration evaluation • Add `monotonicity` test for model behavior consistency checks • Add `smoothness` test for output stability analysis Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.9.0 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.9.0 Mon, 08 Apr 2024 14:07:37 GMT Giskard v2.9.0 adds multi-mlworker support, kernel auto-generation, and managed mlworker lifecycle methods • Adds ability to connect multiple external ML workers via `create_project` method parameters • Introduces automatic kernel generation based on project dependencies • Adds new methods to start and stop managed ML workers Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.7.5 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.7.5 Wed, 06 Mar 2024 12:27:39 GMT Adds API for executing test suites and uploading results to the Hub • Integrates Sentry for error tracking Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.7.4 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.7.4 Wed, 28 Feb 2024 15:11:29 GMT Giskard v2.7.4 adds deterministic seeding to LLM generators and evaluators for reproducible test results. • Adds `seed` parameter to LLM generators and evaluators to ensure reproducibility Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.7.3 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.7.3 Thu, 22 Feb 2024 16:39:31 GMT Giskard v2.7.3 adds llm-as-a-judge per-row metadata requirements and fragment markdown report support. • llm-as-a-judge tests can now include per-row test metadata specifying `requirements` Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.7.2 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.7.2 Mon, 19 Feb 2024 10:51:17 GMT Adds RAG toolset for evaluating retrieval-augmented generation pipelines • Introduces the `RAGToolset` class to evaluate Retrieval-Augmented Generation models Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.7.1 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.7.1 Tue, 13 Feb 2024 15:20:44 GMT Giskard v2.7.1 adds Number-to-Words transformation and LLM tool calls support • Added Number-to-Words Transformation as a new transform capability. • Added tool calls support for LLM clients. • Added scan Transformation Function to the Hub. Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.7.0 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.7.0 Mon, 29 Jan 2024 17:37:44 GMT Giskard v2.7.0 adds LLM-as-judge ground truth tests and custom scanners • Introduces the ability to define a custom scanner via the `CustomScanner` class Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.6.0 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.6.0 Tue, 23 Jan 2024 20:44:59 GMT Adds to_junit() method and improved `__str__` for Dataset/Model in v2.6.0 • Adds a `to_junit` method to `TestSuiteResult` to export reports in JUnit XML format • Updates `Dataset.__str__` and `Model.__str__` to display the object's name and ID instead of default representation Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.5.3 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.5.3 Fri, 19 Jan 2024 15:17:07 GMT Giskard v2.5.3 adds a method to create Dataset from TestDetails and includes ggshield in pre-commit. • Adds Dataset.from_test_details() method to construct datasets directly from test results • Adds `ggshield` to the project's pre-commit configuration for secret scanning Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.3.0 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.3.0 Thu, 04 Jan 2024 11:08:35 GMT Giskard v2.3.0 allows cancelling Hub jobs and refactors ML worker code • Adds the ability to cancel Hub jobs via API or UI (JIRA `GSK-1943`) Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.2.0 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.2.0 Wed, 03 Jan 2024 14:49:34 GMT Giskard v2.2.0 adds speech-to-text typo transformation and refactors prompt injection detection. • Add speech-to-text typos transformation • Complete refactoring of prompt injection Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.1.3 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.1.3 Thu, 21 Dec 2023 09:18:06 GMT Adds LLM-as-judge per-row evaluation and data quality tests • Added `llm_as_judge` row evaluator for evaluating model outputs against an LLM criterion per row • Added new `data_quality` tests to assess dataset integrity Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.1.2 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.1.2 Mon, 18 Dec 2023 17:06:36 GMT Giskard v2.1.2 adds Fisher's exact test and permutation tests for slice metric significance, plus explicit cache persistence control. • Adds Fisher's exact test and permutation test support for slice metrics significance testing • Adds explicit control over when to persist/read the internal cache via a new configuration option Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.0.6 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.0.6 Sun, 03 Dec 2023 21:39:21 GMT Adds feature subset scanning, Azure LLM scan support, and `metric` field to LLM issues in v2.0.6 • Enables running LLM scans using Azure OpenAI • Includes a `metric` field in LLM issue objects Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.0.4 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.0.4 Thu, 23 Nov 2023 00:15:51 GMT Adds AVID integration to Giskard for model auditing reports. • AVID integration Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.0.3 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.0.3 Wed, 15 Nov 2023 13:23:38 GMT Giskard v2.0.3 tracks LLM calls/tokens in scans and requires name/desc for text_generation • Adds `scan` tracking for the number of LLM calls and tokens used • Makes `name` and `description` mandatory parameters in `text_generation` functions Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.0.0 https://github.com/Giskard-AI/giskard-oss/releases/tag/v2.0.0 Mon, 06 Nov 2023 13:44:54 GMT Giskard v2.0 introduces correlation and spurious correlation detectors, MLflow integration, and WebSocket-based worker status updates. • `CorrelationDetector` available for detecting feature correlations in datasets • `SpuriousCorrelationDetector` added to the test suite for identifying spurious correlations • Integration with `MLflow` for experiment tracking and artifact logging • Worker backend communication switched to WebSockets, enabling reactive worker status updates in the UI Full history on Cyber Toolchain: https://cybertoolchain.io/tools/giskard