Outcome: Generate a shopping list for three different vendors by uploading a photo of our inventory worksheet.

Solution: Use a custom GPT to train on simple data and take advantage of the built-in photo recognition (OCR).

Maintaining a well stocked kitchen for a coffee shop is something I underestimated. It’s not complex, but it is very detail oriented and can have a lot of steps. Having an item out of stock can create a poor customer experience, and if the customer came just for that item, they might not come back.

As a small cafe we don’t have a single major suppler, I drive to three different stores to get what we need. Taking inventory for the coffee bar, the kitchen, and paper goods happens every day on two different paper forms that are uploaded to Slack by the employee on shift. For the last year, I would take those sheets and start to make a check list for each store. Depending on the amount of items, this takes 2-10 minutes

I tried to solve this with an off the shelf OCR (Optical character recognition) but the pricing for one that worked well was over $250+/month. I tried to program and train a custom python script using ChatGPT to write the scripting, but the accuracy was too low to be useful. It wasn’t until OpenAI released “GPT’s” that saw the clear path to building this.

Building & Training the GPT#

To build a new GPT, login to ChatGPT and click on Explore GPTs and then click “Create” in the top right corner.

Image Description Give your GPT a name, logo (optional), and short description. Once that is done, you will want to move over to “create” mode by clicking the create tab here:

Image Description This will let you enter into a familiar chatGPT conversational mode where you can describe what you want this GPT to do.

Training the GPT is a very easy process. I explained what outcome I wanted from the GPT and uploaded a dataset in XLS format of all of the possible inventory items the related vendor.

I use Obsidian as my note taking and lists app and it uses Markdown as styling to create lists/formatting. I asked the GPT to only output the list in a markdown codeblock so that it’s easy to copy and paste into Obsidian.

I also have asked it to ask me if I’ve forgotten an item if I send in an inventory sheet missing something from the training data. For example, Whole Milk isn’t listed on the inventory, the GPT is instructed to remind/ask me if we need Whole Milk.

Here is the exact instruction file I use:

You are a coffee shop manager overseeing inventory remotely for a cafe. Your primary goal is to ensure the shop is stocked with all necessary ingredients without having the authority to respond to anything beyond supply checks. Employees report inventory counts, and you identify if anything critical is missing, prompting staff for any overlooked items. 

When responding to reports, ask about any unmentioned items from your internal list before finalizing the supply order. 

you are to only output markdown, formatted in a list format with checkboxes and grouped by the vendor the item is purchased from. The items must be grouped by vendor.

Daily Use#

Image Description

When it’s time to create a shopping list, I download the image of the inventory sheet, login to ChatGPT and select the GPT I created “C&K Inventory Manager.

Image Description

Once uploaded, I may ask it to move some items around if I know I need a bulk item from a larger suppler. Once completed, it outputs Markdown in a code block so it’s easy to copy and pate into Obsidian.

Image Description

If you want to see the final code, here is the markdown output: (Click below to reveal) 


### **Restock List**

#### **Sams Club**
- [ ] Sriracha
- [ ] Cream Cheese

#### **Hannaford**
- [ ] Cinnamon Powder
- [ ] Arugula
- [ ] Bell Peppers
- [ ] Honey
- [ ] AP Flour
- [ ] Vegetable Broth
- [ ] Red Pepper Hummus
- [ ] Sourdough Bread
- [ ] Wheat Bread

#### **Restaurant Depot**
- [ ] Chocolate Chips
- [ ] Red Onions
- [ ] Eggs
- [ ] Chicken Breast

#### **Boston Artisan**
- [ ] Muffins

Updates and taking it further#

With the release of ChatGPT 4o, the logic and multi-step process has increase the accuracy and detail immensely. No changes were made in the instructions or training data. Sometimes it does now overthink and I have to plainly state “just output my markdown list”, as it tried to ask too many questions.

The final form of this system is to integrate this GPT into our Slack channel for ordering. It is currently natively possible to link a Custom GPT to a Slack channel. Once this connection is possible natively or though a custom connector (Zapier, as an example), photos uploaded in our “Ordering” channel will automatically be analyzed and a shopping list sent directly to me split into the correct vendors, areas at each vendor (produce, dairy, etc), and in the correct format (Markdown To-Do List).