Can AI increase productivity and save PMs valuable time?

In a recent survey, more than 1000 product managers were asked whether or not AI tools are advanced enough to be used to write product documents. The consensus? AI tools can be useful for structuring and outlining documents but are not yet accurate enough to create meaningful product documentation without significant input and effort.

Many PMs extolled the virtues of using tools like ChatGPT and Quillbot to write fluid texts that would have otherwise taken hours–but almost all agree that in order for AI to write a meaningful product document, the quality of input is crucial. Without first inputting collected data, research, notes, and product information, AI is likely to produce factual errors such as inaccurate or erratic numbers and fabricated product information.

But is it still worthwhile to use AI to write product documents? Let’s take a closer look.

How to use AI to write product documents

Technically speaking, there are a few different ways that AI can be used to write product documents:

Natural language generation (NLG) systems can be used to automatically generate documents such as user stories or technical documents. However, structured data is needed as input in order for the machine learning algorithms to generate text that is coherent and follows a particular style or format.

AI-powered writing assistants can help PMs generate ideas, organize their thoughts, and improve the overall quality of their writing. These tools also use machine learning algorithms to suggest alternative phrases or rephrasings, and typically also offer grammar and spelling corrections.

AI-powered templates can be used to streamline the document writing process. These templates use machine learning algorithms to suggest the most relevant sections to include in a particular type of document, such as a PRD (product requirements document).

AI-powered summarization tools can be used to automatically generate summaries of longer documents. These tools can be particularly helpful for generating executive summaries or summarizing technical documents for a non-technical audience.

Whichever route you decide to take, be sure to input correct and relevant data and thoroughly review the output before sharing or publishing your product document. AI is not without limitations:

Limitations of AI-written product docs

Whenever you use AI to produce content, it’s important to remember that it’s always “learning” from available data sources, which means it’s also limited to what’s currently out there. Open AI specifically mentions its limitations, including potentially generating incorrect information,  harmful instructions, or biased content. It’s also limited to knowledge of the world and events after 2021.

Moreover, in order for product documentation to be valuable, it needs to include intricate knowledge of your software’s behavior, the problems your product is trying to solve, and the information you learned in user interviews and research. If you want to use AI to write your docs, you’ll need to manually input these details, and you’ll likely spend time editing and tweaking the output.

If you want to see if AI really saves you time, track and see how long it takes you to write a draft vs inputting information into ChatGPT.

Collato puts Open AI to the test

We decided to put our money where our mouth is and test out ChatGPT ourselves to see if it could write a valuable PRD. For our first test, we used Open AI’s dialogue feature and asked it to “please write a PRD for Collato” with no additional input.

The output was lackluster at best–it clearly didn’t fully grasp what Collato was as a tool, using generic terms like “project management software” and “team collaboration tool” (FYI–Collato is actually a search engine for product teams to use to find distributed information across their tools).

Moreover, the text the tool generated was less a comprehensive document and more a simplistic overview of basic requirements of any product, such as “intuitive and easy to use” and “responsive and works well on a variety of devices, including desktop computers, laptops, and mobile phones.”

Basically, it reads like a product requirement document written by someone very new to product management and very unfamiliar with the product itself.

However, once we gave Open AI more context and information by using our own PRD template, we got something a little more interesting and useful:

Example of PRD written by ChatGPT

Collato is a search engine designed specifically for product teams to find information distributed across their various tools. The aim is to save time and increase productivity for teams by providing them with a centralized search function that can quickly retrieve information from all of their relevant tools.

Goal: The goal of Collato is to improve the efficiency of product teams by providing them with a single search engine that can locate distributed information across various tools. The tool aims to improve the productivity of product teams by saving time searching through multiple tools for relevant information.

Stakeholders:

Owner: Product Manager

Development: Engineers, UX Designers

Users: Product teams, including product managers, developers, designers, and other stakeholders involved in product development.

Background:

Many product teams use multiple tools, including project management software, collaboration tools, and various communication channels, to coordinate their work. Searching through multiple tools to find relevant information can be time-consuming and inefficient, leading to reduced productivity and lower quality work.

Strategic fit:

Collato fits into the overall product strategy by supporting the goal of improving productivity and efficiency for product teams. By providing a centralized search engine, product teams can quickly find the information they need, reducing the time spent searching and increasing the time available for other tasks.

Problem Alignment:

The Problem: The problem is that product teams often use multiple tools to manage their work, and searching for relevant information across these tools can be time-consuming and inefficient. This problem can lead to reduced productivity, lower quality work, and missed deadlines.

Desired Behavior:

• Collato will be able to search and retrieve data from different tools and sources like JIRA, GitHub, Confluence, Google Drive, and others, and present the results in a single view.

• The search results will be ranked based on relevance and presented in an easy-to-navigate format that allows users to quickly find what they need.

• The search engine will be customizable and allow users to filter results by tool, date range, and other criteria.

• Collato will support multiple team members searching simultaneously and maintaining their search history for easy reference.

Business Objectives:

Increase user productivity and satisfaction. Differentiate from competitors by offering a unique and comprehensive solution.

Assumptions:

Users will primarily access Collato through desktop computers and laptops. Users will have a variety of experience levels with productivity tools.

User Interaction:

Users will primarily interact with Collato through the visual interface and search functionality

Users can access and edit information and tasks within their own workspaces or those shared with their team

Users can communicate and collaborate with their team through chat and real-time editing

Questions for Further Research:

How can we ensure that the user interface is intuitive and easy to use for users of all experience levels?

What types of integrations would be most valuable to our users?

How can we balance the need for robust security measures with the need for seamless user experience?

Clearly, by providing more contextual input, you’ll get something more meaningful out of AI. The more examples you provide, the more you can train ChatGPT to streamline the production process of writing documents like user stories, PRD, technical docs, API docs, etc.

Tl;dr

AI tools often output generic content without new or interesting information. However, it can also help you write clearer, well-phrased text. The key is to feed it quality information about your product, users, and data and to specify exactly how you want your document formatted. So it might save you time in structuring and drafting your text, but in the end, you will still need to spend time preparing, editing, and reviewing your product documents.

Your role as a PM is to envision the future of the product, which means you need to be able to come up with strong ideas on your own. AI tools like ChatGPT can give you a starting point for structuring your documents, but refining and curating the output into something valuable for your product will always require context and talent on your side.