Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,14 @@ The Flutter Gen UI SDK currently supports A2UI v0.9.

See the [genui getting started guide](packages/genui/README.md#getting-started-with-genui).

## Skills

This repo contains [skill files](packages/genui/skills/) for developers building with agentic coding tools. They can be copied directly into an agent's preferred location or installed using the [`skills`](https://www.npmjs.com/package/skills) package:

```bash
npx skills add https://github.com/flutter/genui/tree/main/packages/genui/skills
```

## Constraints

This repo requires Flutter version >=3.35.7.
Expand Down
125 changes: 125 additions & 0 deletions packages/genui/skills/create-catalog-item/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
---
name: create-catalog-item
description: Use this skill when the user asks to create a new CatalogItem, data class, and/or widget class based on a JSON Schema definition in an application that uses Flutter's `genui` package.
---

# Create CatalogItem

## Goal
To correctly implement a GenUI CatalogItem based on a provided json_schema_builder Schema, including its corresponding data class, CatalogItem instance, and Widget class. This ensures the AI model can properly generate and interact with the UI component.

## Instructions
When tasked with creating a CatalogItem from a `Schema`, follow these steps:

1. **Create the Data Class**:
- Name it `_<SchemaName>Data` (e.g., if schema is `myCardSchema`, data class is `_MyCardData`).
- Add final fields for each property defined in the schema.
- Create a `factory _<SchemaName>Data.fromJson(Map<String, Object?> json)` method.
- Use a `try-catch` block to parse the properties and return a new instance.
- Cast each property from the `json` map to its expected type, e.g., `title: json['title'] as String,` or `action: json['action'] as JsonMap?,`.
- Throw an `Exception('Invalid JSON for _<SchemaName>Data')` in the `catch` block if an error occurs.

2. **Create the CatalogItem Instance**:
- Name it identical to the schema name but without the "Schema" suffix (e.g., `myCard` for `myCardSchema`).
- Declare as a `final CatalogItem`.
- Set `name` to the capitalized version of the name (e.g., `'MyCard'`).
- Set `dataSchema` to the provided schema.
- Implement the `widgetBuilder: (itemContext)`:
- Cast `itemContext.data` to `Map<String, Object?>`.
- Parse the data using the data class `fromJson` method: `_<SchemaName>Data.fromJson(json)`.
- Return the corresponding Widget class and pass the required data.
- If the schema includes an action callback (like `onCompleted`), implement it here. You must parse the action context using `resolveContext` and dispatch an event using `itemContext.dispatchEvent(...)`.

3. **Create the Widget Class**:
- Name it `_<CapitalizedSchemaName>` (e.g., `_MyCard`).
- Inherit from `StatelessWidget` or `StatefulWidget` depending on state requirements.
- Add the Data Class as a required property (e.g., `final _<SchemaName>Data data;`).
- Add any required callback properties (e.g., `final void Function(int) onCompleted;`).
- Implement the `build` method using Flutter Material components (e.g., Card, Column, Text). Make sure each data field in the data class is displayed, and that actions are represented by buttons or other interactive elements.

## Examples
### Input Schema
```dart
final basicCardSchema = S.object(
properties: {
'component': S.string(enumValues: ['BasicCard']),
'title': S.string(),
'action': A2uiSchemas.action(),
},
required: ['title'],
);
```

### Expected Output
```dart
class _BasicCardData {
final String title;
final JsonMap? action;

_BasicCardData({required this.title, this.action});

factory _BasicCardData.fromJson(Map<String, Object?> json) {
try {
return _BasicCardData(
title: json['title'] as String,
action: json['action'] as JsonMap?,
);
} catch (e) {
throw Exception('Invalid JSON for _BasicCardData: $e');
}
}
}

final basicCard = CatalogItem(
name: 'BasicCard',
dataSchema: basicCardSchema,
widgetBuilder: (itemContext) {
final json = itemContext.data as Map<String, Object?>;
final data = _BasicCardData.fromJson(json);

return _BasicCard(
data: data,
onTap: () async {
final action = data.action;
if (action == null) return;
final event = action['event'] as JsonMap?;
final name = (event?['name'] as String?) ?? '';
final JsonMap contextDefinition =
(event?['context'] as JsonMap?) ?? <String, Object?>{};
final JsonMap resolvedContext = await resolveContext(
itemContext.dataContext,
contextDefinition,
);
itemContext.dispatchEvent(
UserActionEvent(
name: name,
sourceComponentId: itemContext.id,
context: resolvedContext,
),
);
}
);
},
);

class _BasicCard extends StatelessWidget {
final _BasicCardData data;
final VoidCallback onTap;

const _BasicCard({super.key, required this.data, required this.onTap});

@override
Widget build(BuildContext context) {
return Card(
child: ListTile(
title: Text(data.title),
onTap: onTap,
),
);
}
}
```

## Constraints
- Ensure proper use of `try-catch` blocks and type casting when parsing JSON in `fromJson`.
- Make sure action resolution accurately fetches variables via `resolveContext` and uses `itemContext.dispatchEvent` when actions are present in the Schema.
151 changes: 151 additions & 0 deletions packages/genui/skills/integrate-genui-firebase/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
---
name: integrate-genui-firebase
description: Use this skill when the user asks to integrate the genui package and get a simple conversation going with Firebase AI Logic.
---

# Integrate GenUI with Firebase AI Logic

## Goal
To successfully integrate the `genui` package into a Flutter app and set up a basic conversational agent using Firebase AI Logic. This skill assumes Firebase AI Logic is already set up and working in the project.

## Instructions
When tasked with integrating `genui` and starting a simple conversation, follow these steps:

1. **Verify Firebase Setup:**
Ensure `firebase_core` and `firebase_ai` are available in `pubspec.yaml`.
Verify that `Firebase.initializeApp` is called in the `main()` function:
```dart
WidgetsFlutterBinding.ensureInitialized();
await Firebase.initializeApp(options: DefaultFirebaseOptions.currentPlatform);
```

2. **Add GenUI Package:**
Add `genui` to the `pubspec.yaml` dependencies.

3. **Import Required Libraries:**
Import `genui` and hide `TextPart` so it doesn't conflict with other packages, then import it again with an alias:
```dart
import 'package:genui/genui.dart' hide TextPart;
import 'package:genui/genui.dart' as genui;
```

4. **Configure Basic Logging:**
At the beginning of the `main()` function, configure GenUI logging:
```dart
configureLogging(
logCallback: (level, msg) => debugPrint('GenUI $level: $msg'),
);
```

5. **Create Model and Chat Session:**
Initialize the generative model and start a chat session.
```dart
final model = FirebaseAI.googleAI().generativeModel(
model: 'gemini-3-flash-preview',
);
final _chatSession = model.startChat();
```

6. **Identify Target StatefulWidget:**
**STOP AND ASK THE USER IF UNCLEAR:** This integration requires a `StatefulWidget` to hold the references to GenUI controllers (`SurfaceController`, `A2uiTransportAdapter`, and `Conversation`). Identify which `StatefulWidget` to use in the application. If you are unsure which widget should hold this state, ask the user before proceeding.

7. **Wire up GenUI Controllers inside State:**
Inside your identified `State` class, instantiate `SurfaceController`, `A2uiTransportAdapter`, and `Conversation`:
```dart
final catalog = BasicCatalogItems.asCatalog(); // Optionally inject custom CatalogItems
final _controller = SurfaceController(catalogs: [catalog]);
final _transport = A2uiTransportAdapter(onSend: _sendAndReceive);
final _conversation = Conversation(
controller: _controller,
transport: _transport,
);
```

8. **Implement the `_sendAndReceive` Method:**
Create a method to take messages from the transport adapter, send them to Firebase, and feed the AI's response back to the transport.
```dart
Future<void> _sendAndReceive(ChatMessage msg) async {
final buffer = StringBuffer();

for (final part in msg.parts) {
if (part.isUiInteractionPart) {
buffer.write(part.asUiInteractionPart!.interaction);
} else if (part is genui.TextPart) {
buffer.write(part.text);
}
}

if (buffer.isEmpty) return;

final text = buffer.toString();
final response = await _chatSession.sendMessage(Content.text(text));

if (response.text?.isNotEmpty ?? false) {
_transport.addChunk(response.text!);
}
}
```

9. **Listen to Conversation Events:**
Create stubbed-out methods in your State class for each event type, including DartDoc comments explaining their required behavior.
```dart
/// Updates state to include the new [surfaceId] so a new `Surface` widget can be built.
void _onSurfaceAdded(String surfaceId) {
// TODO: Implement state update to add surfaceId
}

/// Updates state to remove the [surfaceId] so its `Surface` widget is no longer built.
void _onSurfaceRemoved(String surfaceId) {
// TODO: Implement state update to remove surfaceId
}

/// Handles displaying raw text content received from the AI to the user.
void _onContentReceived(String text) {
// TODO: Implement displaying the received text
}

/// Handles errors that occur during the conversation appropriately.
void _onError(Object error) {
// TODO: Implement error handling
}
```

Subscribe to the `_conversation.events` to track when UI surfaces or chat messages arrive, dispatching them to the appropriate stubbed out methods:
```dart
_conversation.events.listen((event) {
switch (event) {
case ConversationSurfaceAdded added:
_onSurfaceAdded(added.surfaceId);
case ConversationSurfaceRemoved removed:
_onSurfaceRemoved(removed.surfaceId);
case ConversationContentReceived content:
_onContentReceived(content.text);
case ConversationError error:
_onError(error.error);
default:
}
});
```

10. **Initialize System Prompt:**
Use `PromptBuilder` to give the AI basic instructions, then send it as a system message.
```dart
final promptBuilder = PromptBuilder.chat(
catalog: catalog,
instructions: 'You are a helpful assistant. Respond to messages in a chatty way.',
);
_conversation.sendRequest(ChatMessage.system(promptBuilder.systemPrompt));
```

11. **Display Surfaces:**
In your Flutter `build()` method, use the `Surface` widget wherever you need to render GenUI widgets using the `surfaceIds` you collected in step 9.
```dart
Surface(surfaceContext: _controller.contextFor(surfaceId));
```

12. **Ask User for Input Preferences:**
**STOP AND ASK THE USER:** Ask the user for clarification on what UI elements should be used for user input. Explain that a `TextField` and `ElevatedButton` are good defaults, but you should not assume they want those exact widgets unless they clarify.

## Constraints
- Do not make assumptions about user input UI elements; see step 12.
- Make sure to properly clean up GenUI controllers (`_transport.dispose()`, `_controller.dispose()`) inside the widget's `dispose()` method.
Loading