Introduction to Playbooks
Build a library of reusable prompts for your organization
What are Playbooks?
Playbooks are easily shareable, reusable prompts
A playbook is like a custom system prompt. It’s also like writing a program, without the rigid syntax.
Playbooks are also easily shareable and reusable, so once anyone succeeds with Devin, others can more easily replicate that success!
We recommend using Playbooks when:
- You or your teammates will reuse the prompt on multiple sessions.
- You find yourself repeating the same reminders to Devin
- The use case may be relevant to others — in your organization or within the Devin user community.
Getting Started with Playbooks
Consider writing your first playbook with a simple multi-step task you want Devin to tackle.
- Create a document that outlines…
- The outcome you want Devin to achieve
- The steps required to get there
- Optional: Add sections like Procedure, Specifications, Advice, Forbidden Actions or Required from User
- Procedure: Outline the entire scope of the task. Include at least one step for setup, the actual task, and delivery.
- Specifications: Describe postconditions - what should be true after Devin is done?
- Advice: Include tips to correct Devin’s priors
- Forbidden Actions: Include any action Devin should absolutely not take
- Required from User: Describe any input or information required from the user
- Create the playbook directly in the web app by clicking “Create a new Playbook”. Alternatively, save a file with the file extension
.devin.md
and drag-and-drop it in the web app when starting a Devin session
Writing a Great Playbook
## Procedure
The Procedure section should…
- Have one step per line, each line written imperatively
- Cover the entire scope of the task
- Include at least one step for setup, the actual task, and delivery
- Aim to make the steps Mutually Exclusive and Collectively Exhaustive
- Additional Tips
- Procedures should help you define the order of Devin’s action - like if/else/loops/goto in code
- Don’t make tasks too specific unless you really need to, this can reduce Devin’s ability to problem-solve
- Each procedure step should contain an action verb - e.g. Write, Navigate to, etc.
## Advice and Pointers
Share advice and pointers with Devin if…
- You have a preferred way of completing the tasks
- The advice applies to the entire task, or multiple steps. Advice specific to one step should be written next to that step (e.g. as a sub-bullet)
- You are correcting Devin’s priors. Advice can function like comments on pseudocode that influence its execution.
## Specifications
The Specifications section can be helpful to describe the postconditions of the playbook — what should be true once Devin is done?
## What’s Needed From User
Think through anything necessary but outside of Devin’s control. For example, if the user needs to provide a token or information that is not publicly available to Devin.
Other Tips + Tactics
Example Playbook
View example sessions using the playbook below here and here.
Playbook: R Data Science Tutorial
## Overview
Create a data science tutorial using an R markdown notebook.
## What’s Needed From User
- Link to a dataset (csv file attachment or kaggle link)
- Specific task to create a data science tutorial for
## Procedure
1. Download the dataset provided by the user.
- If needed, download the dataset using the Kaggle CLI - you don't need any credentials for this
2. Create an R markdown notebook titled `data_science_tutorial.Rmd`.
3. Create a `tmp.Rmd` file for writing and saving intermediate code.
4. Create 5 main sections inside the `data_science_tutorial.Rmd` file and add code from the `tmp.Rmd` file containing the following:
- Dataset Statistics. Generate a statistical summary of the dataset.
- EDA (Exploratory Data Analysis). Create a bar chart and a scatter plot for the provided data.
- Train-test split. Split the data in an 80:20 ratio. Save the training and testing data.
- Training the machine learning model. Save the model once trained.
- Inference with the saved model. Load the saved model and evaluate its performance on the test set using the metric specified by the user.
5. Once the code is written, add a short explanation for each section.
6. Convert the R markdown notebook to HTML format
7. Send the final R markdown notebook, HTML file, saved model and testing data to the user.
## Specifications
1. Send the R markdown notebook and HTML file to the user.
2. Send the saved model and testing data to the user.
## Advice and Pointers
1. Do not to re-install packages if already installed.
2. Sign in to RStudio is not required to complete this task.
3. Run the entire notebook after you add code for each section.
## Forbidden Actions
1. Do not overwrite the `data_science_tutorial.Rmd` file.