Are AI tools advanced enough for product documentation?
Using AI to write documentation: What you should know
Time is of the essence in product management. PMs are constantly seeking innovative ways to increase productivity and save valuable time. Nowadays, there are tons of AI-powered products that seek to streamline your workday. Enter the era of AI tools for documentation–but is AI a game-changer that promises to revolutionize the way PMs create impactful product documents? Or is AI a time-waster with empty results?
Many PMs are already using tools like ChatGPT or Bard to help them write fluid texts that would have otherwise taken hours–but almost all agree that in order for AI to write a meaningful product document, the quality of input is crucial. Without first inputting collected data, research, notes, and product information, AI is likely to produce factual errors such as inaccurate or erratic numbers and fabricated product information.
That’s why we decided to make Collato the way it is today. By first connecting your tools (Confluence, Jira, Google Drive, etc), you can use AI to write comprehensive and accurate documents, all sourced from your product info without worrying if it is hallucinating.
AI has already begun to transform the way we work. But can it really write quality documents?
Let’s take a closer look.
How to use AI to write product documentation
Technically speaking, there are a few different ways for AI to write product documentation:
Natural language generation (NLG) systems can be used to automatically generate documents such as user stories or technical documents. However, structured data is needed as input in order for the machine learning algorithms to generate text that is coherent and follows a particular style or format.
AI-powered writing assistants can help PMs generate ideas, organize their thoughts, and improve the overall quality of their writing. These tools also use machine learning algorithms to suggest alternative phrases or rephrasings, and typically also offer grammar and spelling corrections.
AI-powered templates can be used to streamline the document writing process. These templates use machine learning algorithms to suggest the most relevant sections to include in a particular type of document, such as a PRD (product requirements document).
AI-powered summarization tools can be used to automatically generate summaries of longer documents. These tools can be particularly helpful for generating executive summaries or summarizing technical documents for a non-technical audience.
Collato is basically a combination of all of these options: It’s an AI Assistant designed to find, summarize, and generate new content based on existing content that lives in your current tool stack. It also has access to public knowledge, similar to ChatGPT and Bard.
To use Collato to write your documentation, first connect your tools. You can upload single files, or auto-sync your entire Notion, Confluence, Jira, or Google Drive.
Next, prompt Collato. Ask something like, “Write a PRD for [specific feature] using the same format as [a previous PRD].” Pulling from your meeting minutes, research notes, and any other relevant information, Collato’s AI Assistant will have your document ready in seconds.
If you use Collato to write your product documentation, the AI assistant will pull the most relevant data that you’ve synced to the platform in order to write whatever document type you need, in whatever format you prefer.
Want to double-check that the information inside is accurate? No problem. Collato lists its sources in each response, which you can click on and view inside the tool. That way, you can easily double-check if something is right, or read more about a certain topic.
Limitations of AI-written product documentation
Whenever you use other AI tools (like ChatGPT) to produce content, it’s important to remember that it’s always “learning” from available data sources, which means it’s using your data to train itself. This also means that your information is potentially available to anyone else who might ask ChatGPT–so be careful what input you give it. It’s also limited to what’s currently out there. Open AI specifically mentions its limitations, including potentially generating incorrect information, harmful instructions, or biased content. It’s also limited to knowledge of the world and events after 2021.
If you use Collato to write your documentation, your data is never used to train AI. Collato is SOC 2 certified, GDPR compliant, and hosted in Germany–meaning your data is completely secure and private.
In order for product documentation to be valuable, it needs to include intricate knowledge of your software’s behavior, the problems your product is trying to solve, and the information you learned in user interviews and research. If you want to use other AI tools to write your docs, you’ll need to manually input these details, and you’ll likely spend time editing and tweaking the output. If you use Collato, you don’t need to worry about that–it’ll refer to the relevant information and pull from that to write your document.
Collato puts ChatGPT to the test
We decided to test out ChatGPT ourselves to see if it could write a valuable PRD. For our first test, we used Open AI’s dialogue feature and asked it to “please write a PRD for Collato” with no additional input.
The output was lackluster at best–it clearly didn’t fully grasp what Collato was as a tool, using generic terms like “project management software” and “team collaboration tool” (FYI–Collato is actually an AI Assistant for product teams).
Moreover, the text the tool generated was less a comprehensive document and more a simplistic overview of basic requirements of any product, such as “intuitive and easy to use” and “responsive and works well on a variety of devices, including desktop computers, laptops, and mobile phones.”
Basically, it reads like a product requirement document written by someone very new to product management and very unfamiliar with the product itself.
Example of documentation written by Collato
Time to put our money where our mouth is: We put Collato to the test to see if it could write a PRD for Dots, a fictional product with fictional documentation that we created to test out our product. We prompted Collato to write a PRD for Dots, and here is the result:
Purpose: The purpose of the Dots product is to provide a fun and engaging way for users to connect with each other.
Overview: Dots is a social media platform that allows users to share short videos, photos, and text posts with their friends and followers. Users can also follow other users, like posts, and comment on posts.
Features and functionalities:
The key features of Dots include:
- Sharing short videos, photos, and text posts
- Following other users
- Liking posts
- Commenting on posts
- Sending private messages
- Receiving notifications
Prioritized features: The most important features of Dots are sharing content, following other users, and liking posts. These features are essential for users to be able to connect with each other and share their thoughts and feelings.
Here are some user stories for Dots:
- As a user, I want to be able to share short videos, photos, and text posts with my friends and followers.
- As a user, I want to be able to follow other users so that I can see their content in my feed.
- As a user, I want to be able to like posts so that I can show my appreciation for the content.
The technical specifications for Dots include:
- The platform will be hosted on a cloud-based server.
- The platform will use a secure database to store user data.
- The platform will use a content delivery network (CDN) to deliver content to users.
Assumptions and constraints:
The following assumptions and constraints apply to Dots:
- Users have access to the internet.
- Users have a device that can access the internet.
- Users have a social media account.
The following potential risks exist for Dots:
- Users may post inappropriate content.
- Users may be targeted by spam or phishing attacks.
- Users may lose their data if the platform is hacked.
Milestones and timeline:
The following milestones and timeline are planned for Dots:
Q1 2023: Launch the platform in beta.
Q2 2023: Add new features and functionality.
Q3 2023: Launch the platform in full production.
For a fictional PRD for a fictional company–it’s a pretty comprehensive as a document! As for sources used, Collato lists feature specs, the company’s vision and mission statement, and a market research document.
Generate content based on your product information
One of the challenges with using AI tools for documentation and content creation is the reliance on input from unknown or untrustworthy sources that can often be broad, unspecific, or irrelevant to your team's needs.
Collato, an AI assistant designed specifically for product teams, works exclusively with your team's information. Use it to find, summarize, or generate content based on your team’s knowledge.
By incorporating only the files you choose, Collato ensures that the content it generates, whether it's blog posts or tweets, is based on your team's unique knowledge and expertise. Collato now integrates with Slack, making your team's valuable knowledge easily accessible right within the app. Now, you can enjoy seamless collaboration, instant access to vital information, and a boost in productivity without interrupting your workflow.
AI tools often output generic content without new or interesting information. However, it can also help you write clearer, well-phrased text. The key is to feed it quality information about your product, users, and data and to specify exactly how you want your document formatted. Tools like Collato will save you time and ensure that your documents are created specifically for your products’ needs.
Your role as a PM is to envision the future of the product, which means you need to be able to come up with strong ideas on your own. AI tools can be a great starting point for structuring your documents, but refining and curating the output into something valuable for your product will always require context and talent on your side. Take back the time you saved writing the document to focus on your vision, strategy, and creating a valuable product as efficiently as possible.