Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,50 @@
# Changelog

All notable changes to this project will be documented in this file.
## [0.8.0] - 2026-05-19

### Features

- openai sts
- add content wrapper to tester
- add content wrapper to generator and plugins. Updated module description and option type hints.
- digraphic lang and updated module tags (#91)
- aws transcribe stt + linting + aws profiles to polly
- add provider timeouts (#95)
- bedrock sso provider
- implement streaming for TTS providers
- add aws polly tts provider
- add TTS plugin
- add elevenlabs tts and stt providers
- add openai tts and stt providers
- add text2image plugin
- Suppress warnings during model loading in OpusTranslator
- Update attack module tags
- Improve plugin module tag taxonomies and list representations
- add digraphic language plugin

### Fixes

- typing of static multi-turn dataset generation
- judge list flattening bug
- tester original attack input error
- list bug
- process_conversation bug
- handle plugin transformation errors and improve AWS credential handling
- async any-llm bug
- content wrapper generator bug
- provider hinting
- generation content, update tests
- update default model IDs

### Changes

- minimised content wrappers
- fix type hinting and add content type checks for providers
- move profiles to any-llm bedrock provider
- update naming scheme and add docs
- Enhance error handling in spikee list and add multi-modal tags

## [0.7.2] - 2026-04-08

### Fixes
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
</p>
</div>

_Version: 0.7.3-dev_
_Version: 0.8.1-dev_


Developed by Reversec Labs, `spikee` is a toolkit for assessing the resilience of LLMs, guardrails, and applications against prompt injection and jailbreaking. Spikee's strength is its modular design, which allows for easy customization of every part of the testing process.
Expand Down
4 changes: 4 additions & 0 deletions docs/14_functional_testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,10 @@ Spikee ships with an end-to-end functional suite that exercises the CLI exactly
- Bootstrap a scratch workspace (`spikee init`) and overlay the fixtures under `tests/functional/fixtures`.
- Execute the relevant `spikee` CLI commands (currently `spikee generate` and `spikee test`) and assert the outputs.

Optionally, run the functional suite from your workspace `pytest ../tests/functional` to use enviromental variables from your workspace .env file:
- `SPIKEE_TESTS_USE_ISOLATED_VENV=1` - Uses current environment instead of creating a new one for each test session. This is useful if you have already installed spikee in your current environment and want to speed up the tests by skipping the installation step.
- Uses LLM provider inference keys.

4. **Run a single test** (useful while debugging):

```bash
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "spikee"
version = "0.7.3-dev"
version = "0.8.1-dev"
description = "Spikee - Simple Prompt Injection Kit for Evaluation and Exploitation"
readme = "README.md"
keywords = [ "prompt-injection", "LLM", "cyber-security", "pentesting",]
Expand Down
2 changes: 1 addition & 1 deletion spikee/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = "0.7.3-dev"
__version__ = "0.8.1-dev"
17 changes: 12 additions & 5 deletions spikee/attacks/anti_spotlighting.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,12 @@

from spikee.tester import AdvancedTargetWrapper
from spikee.templates.attack import Attack
from spikee.utilities.hinting import ModuleDescriptionHint, ModuleOptionsHint, AttackResponseHint, process_target_content
from spikee.utilities.hinting import (
ModuleDescriptionHint,
ModuleOptionsHint,
AttackResponseHint,
process_target_content,
)
from spikee.utilities.enums import ModuleTag


Expand Down Expand Up @@ -75,7 +80,9 @@ def attack(
"""
original_text = entry.get("content", entry.get("text", ""))
if entry.get("content_type", "text") != "text":
raise ValueError("Anti-Spotlighting Attack only supports text content type.")
raise ValueError(
"Anti-Spotlighting Attack only supports text content type."
)
payload = entry.get("payload", None)
system_message = entry.get("system_message", None)
last_payload = original_text # fallback if no transformation occurs
Expand All @@ -101,9 +108,9 @@ def attack(
last_payload = candidate_text

try:
response = process_target_content(target_module.process_input(
candidate_text, system_message
))
response = process_target_content(
target_module.process_input(candidate_text, system_message)
)

last_response = response
success = call_judge(entry, response)
Expand Down
15 changes: 10 additions & 5 deletions spikee/attacks/best_of_n.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,12 @@

from spikee.tester import AdvancedTargetWrapper
from spikee.templates.attack import Attack
from spikee.utilities.hinting import ModuleDescriptionHint, ModuleOptionsHint, AttackResponseHint, process_target_content
from spikee.utilities.hinting import (
ModuleDescriptionHint,
ModuleOptionsHint,
AttackResponseHint,
process_target_content,
)
from spikee.utilities.enums import ModuleTag


Expand Down Expand Up @@ -82,9 +87,9 @@ def attack(
last_payload = candidate_text

try:
response = process_target_content(target_module.process_input(
candidate_text, system_message
))
response = process_target_content(
target_module.process_input(candidate_text, system_message)
)
last_response = response
success = call_judge(entry, response)
except Exception as e:
Expand Down Expand Up @@ -119,7 +124,7 @@ def _scramble_payload_only(self, original_text, payload, exclusions):
return (
original_text[:idx]
+ scrambled_payload
+ original_text[idx + len(payload):]
+ original_text[idx + len(payload) :]
)

def _generate_variant(self, text, exclusions):
Expand Down
33 changes: 22 additions & 11 deletions spikee/attacks/crescendo.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,12 @@
from spikee.tester import AdvancedTargetWrapper
from spikee.templates.attack import Attack
from spikee.templates.standardised_conversation import StandardisedConversation
from spikee.utilities.hinting import ModuleDescriptionHint, ModuleOptionsHint, AttackResponseHint, process_target_content
from spikee.utilities.hinting import (
ModuleDescriptionHint,
ModuleOptionsHint,
AttackResponseHint,
process_target_content,
)
from spikee.utilities.enums import Turn, ModuleTag
from spikee.utilities.llm import get_llm
from spikee.utilities.llm_message import HumanMessage
Expand Down Expand Up @@ -165,7 +170,7 @@ def _extract_json_or_fail(self, text: str) -> Dict[str, Any]:
if depth > 0:
depth -= 1
if depth == 0 and start != -1:
candidate = quoted[start: i + 1]
candidate = quoted[start : i + 1]
try:
return json.loads(candidate)
except Exception:
Expand All @@ -192,7 +197,7 @@ def _extract_json_or_fail(self, text: str) -> Dict[str, Any]:
if depth > 0:
depth -= 1
if depth == 0 and start != -1:
candidate = t[start: i + 1]
candidate = t[start : i + 1]
try:
return json.loads(candidate)
except Exception:
Expand Down Expand Up @@ -318,12 +323,14 @@ def _run_attempt(
)

# call target; each call consumes budget immediately
response_text = process_target_content(target_module.process_input(
prompt,
system_message,
spikee_session_id=session_id,
backtrack=is_refuse,
))
response_text = process_target_content(
target_module.process_input(
prompt,
system_message,
spikee_session_id=session_id,
backtrack=is_refuse,
)
)

last_message_id = conversation.add_message(
last_message_id,
Expand Down Expand Up @@ -395,7 +402,9 @@ def attack(
- conversation is the chat history list (may be empty).
"""
if "content" not in entry and "text" not in entry:
raise ValueError("entry must contain 'content' or 'text' (the jailbreak objective)")
raise ValueError(
"entry must contain 'content' or 'text' (the jailbreak objective)"
)

objective = entry.get("content", entry.get("text", ""))

Expand All @@ -407,7 +416,9 @@ def attack(
max_turns = self._parse_max_turns(opts)
model = opts.get("model", None)
if not model:
raise ValueError("Crescendo attack requires an LLM model specified in options, e.g. 'model=gpt-4'")
raise ValueError(
"Crescendo attack requires an LLM model specified in options, e.g. 'model=gpt-4'"
)

llm = get_llm(model, max_tokens=None)

Expand Down
35 changes: 24 additions & 11 deletions spikee/attacks/echo_chamber.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,12 @@
from typing import List, Callable, Tuple, Union
import uuid

from spikee.utilities.hinting import ModuleDescriptionHint, ModuleOptionsHint, AttackResponseHint, process_target_content
from spikee.utilities.hinting import (
ModuleDescriptionHint,
ModuleOptionsHint,
AttackResponseHint,
process_target_content,
)
from spikee.utilities.enums import ModuleTag, Turn
from spikee.templates.attack import Attack
from spikee.templates.standardised_conversation import StandardisedConversation
Expand Down Expand Up @@ -593,9 +598,11 @@ def get_seed_sentences(
+ json.dumps({"keywords": keywords})
)

response = process_target_content(target.process_input(
prompt, spikee_session_id=spikee_session_id, backtrack=False
))
response = process_target_content(
target.process_input(
prompt, spikee_session_id=spikee_session_id, backtrack=False
)
)

try:
seed_sentences = extract_json_or_fail(response).get("sentences", None)
Expand Down Expand Up @@ -630,7 +637,9 @@ def attack(
tuple: (iterations_used:int, success:bool, {"objective": str, "conversation": List[Dict]}, last_response:str)
"""
if "text" not in entry and "content" not in entry:
raise ValueError("entry must contain 'text' or 'content' (the jailbreak objective)")
raise ValueError(
"entry must contain 'text' or 'content' (the jailbreak objective)"
)

objective = entry.get("content", entry.get("text", ""))

Expand All @@ -644,7 +653,9 @@ def attack(
self.debug = opts.get("debug", "false").lower() == "true"
model = opts.get("model", None)
if not model:
raise ValueError("Echo Chamber attack requires a 'model' option specifying the LLM to use.")
raise ValueError(
"Echo Chamber attack requires a 'model' option specifying the LLM to use."
)

llm = get_llm(model, max_tokens=None, temperature=1)
agent = get_llm(model, max_tokens=None, temperature=0.7)
Expand Down Expand Up @@ -917,11 +928,13 @@ def attack(
attempt=True,
)

last_response = process_target_content(target_module.process_input(
prompt,
spikee_session_id=spikee_session_id,
backtrack=backtrack,
))
last_response = process_target_content(
target_module.process_input(
prompt,
spikee_session_id=spikee_session_id,
backtrack=backtrack,
)
)
backtrack = False

except GuardrailTrigger as e:
Expand Down
17 changes: 12 additions & 5 deletions spikee/attacks/llm_jailbreaker.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,12 @@

from spikee.tester import AdvancedTargetWrapper
from spikee.templates.attack import Attack
from spikee.utilities.hinting import ModuleDescriptionHint, ModuleOptionsHint, AttackResponseHint, process_target_content
from spikee.utilities.hinting import (
ModuleDescriptionHint,
ModuleOptionsHint,
AttackResponseHint,
process_target_content,
)
from spikee.utilities.enums import ModuleTag
from spikee.utilities.llm import get_llm
from spikee.utilities.llm_message import HumanMessage
Expand Down Expand Up @@ -188,10 +193,12 @@ def attack(
)

# Send the attack prompt to the target
last_response = process_target_content(target_module.process_input(
attack_prompt,
entry.get("system_message", None),
))
last_response = process_target_content(
target_module.process_input(
attack_prompt,
entry.get("system_message", None),
)
)

# Add this attempt to our history
previous_attempts.append(
Expand Down
21 changes: 15 additions & 6 deletions spikee/attacks/llm_multi_language_jailbreaker.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,12 @@

from spikee.tester import AdvancedTargetWrapper
from spikee.templates.attack import Attack
from spikee.utilities.hinting import ModuleDescriptionHint, ModuleOptionsHint, AttackResponseHint, process_target_content
from spikee.utilities.hinting import (
ModuleDescriptionHint,
ModuleOptionsHint,
AttackResponseHint,
process_target_content,
)
from spikee.utilities.enums import ModuleTag
from spikee.utilities.llm import get_llm
from spikee.utilities.llm_message import HumanMessage
Expand Down Expand Up @@ -149,7 +154,9 @@ def attack(
# Get the objective from the entry
objective = entry.get("content", entry.get("text", ""))
if entry.get("content_type", "text") != "text":
raise ValueError("LLMMultiLanguageJailbreaker Attack only supports text content type.")
raise ValueError(
"LLMMultiLanguageJailbreaker Attack only supports text content type."
)

if not objective:
return 0, False, "", "No objective provided in entry"
Expand All @@ -172,10 +179,12 @@ def attack(
)

# Send the attack prompt to the target
last_response = process_target_content(target_module.process_input(
attack_prompt,
entry.get("system_message", None),
))
last_response = process_target_content(
target_module.process_input(
attack_prompt,
entry.get("system_message", None),
)
)

# Add this attempt to our history
previous_attempts.append(
Expand Down
Loading