Week 5 of 6
Week 5 90 minutes

AI Advanced

Agents and automation for autonomous operations

AI Advanced

What you will learn

  • Understand what AI agents can and cannot do
  • Choose the right agent tool for different task types
  • Apply web scraping ethics and best practices
  • Design middleware workflows with AI steps
  • Decide when to use agents vs workflows vs MCP

Topics covered

AI Agent Capabilities Agent Tools Comparison Web Scraping Ethics Middleware Workflows Platform Comparison Decision Framework

This week escalates from AI-assisted work to AI-automated work. The difference: AI assistance requires your involvement for every task. AI automation runs independently, 24/7.

What are AI agents?

AI agents are software that mimics human actions on websites and apps. They navigate, make decisions, click, type, and complete processes autonomously.

Core capabilities

Planning: Given a goal, agents plan all the steps required to achieve it. You describe the outcome; they figure out the path.

Actions: Within a plan, agents perform actions like clicking buttons, filling forms, and navigating between pages. They re-plan when they encounter unexpected situations.

Reading and research: Agents analyze unpredictable web pages and adjust their plans in real-time based on what they find.

What this enables

  • Extract structured data from websites (web scraping)
  • Navigate and interact with any web interface
  • Make decisions about next steps based on context
  • Handle variations and edge cases intelligently

Real use case example

Build an automated lead generation system:

  1. Research competitor customer lists from public web pages
  2. Gather business info and identify decision makers
  3. Extract contact information
  4. Format into a structured lead list
  5. Automatically add to your CRM

This would take hours of manual work. An agent can do it while you sleep.

AI agent tools comparison

ToolBest forStrengths
Claude for ChromeResearch across websites, building lead lists, tasks needing judgmentMost intelligent, best at handling variations
OpenAI OperatorMulti-step tasks like bookings and purchasesPowerful, handles complex sequences
BardeenPre-recorded templates, LinkedIn and CRM tasksGood for repetitive, templated actions
Browse AIScheduled tasks running without supervisionEasy template creation, reliable scheduling

Choosing the right tool

Need judgment and intelligence? Claude for Chrome handles variations best.

Complex multi-step sequences? OpenAI Operator excels at chained actions.

Repetitive templated tasks? Bardeen or Browse AI are more efficient.

Web scraping ethics and best practices

Generally acceptable

  • Public business leads
  • Published contact information
  • Product information and pricing
  • Your own data from platforms you use

Avoid (often illegal or unethical)

  • Personal data without consent
  • Content behind logins or paywalls
  • Ignoring website terms of service
  • Mass data collection at aggressive rates

The golden rule

Treat automated access the way you would want to be treated if someone scraped your site. Be reasonable about volume, respect rate limits, and do not take data that clearly is not meant to be public.

Middleware: Automated workflows connecting apps

Middleware platforms let you create automated workflows across various apps. They are the glue that connects your tools without manual intervention.

Simple workflow example

Trigger: New form submission on website

Then automatically:

  1. Add to Google Drive spreadsheet
  2. AI drafts personalized reply using context
  3. Send email via Gmail
  4. Create task in CRM for follow-up

All happens automatically, 24/7, without you touching anything.

How workflows work

Steps can execute:

  • In order: Sequential processing where each step waits for the previous
  • In parallel: Simultaneous actions that do not depend on each other
  • Conditionally: Only if certain criteria are met

The power is in combining these patterns with AI steps that add intelligence to otherwise mechanical processes.

Middleware platform comparison

AspectZapierMaken8n
Ease of startEasiestModerateSteeper learning curve
CostMost expensiveBetter valueMost cost-effective
AI featuresGood coverageGood coverageStrong via community
Visual builderSimpleGoodExcellent
Complex logicLimitedFlexibleExcellent
Self-hostingNoNoYes

Decision guidance

Start with Zapier if you want simplicity and are willing to pay for it.

Move to Make when you need more power at lower cost.

Consider n8n when you want maximum control, can handle the learning curve, or want to self-host.

Decision framework: MCP vs workflows vs agents

Different tools for different jobs:

Use workflows when

  • The step is repetitive and predictable
  • It needs to run 24/7 without supervision
  • The same steps happen every time
  • You want guaranteed, predictable results
  • Example: Every order triggers the same sequence - add to database, send confirmation, create fulfillment task

Use MCP when

  • The task requires analysis or judgment
  • You are asking questions about unstructured data
  • The approach varies based on context
  • You are involved in the conversation
  • Example: “Which customers should I reach out to this week?” requires thinking, not automation

Use AI agents when

  • You need to click or type on websites
  • You are extracting data from web pages
  • You are navigating complex, unpredictable interfaces
  • Example: Build lead lists by researching competitor websites with varying structures

Use traditional web scraping when

  • Simple data extraction from structured pages
  • No clicking or typing needed
  • High volume, low complexity
  • Tools: Browse AI, Instant Data Scraper, Octoparse

Best practices for AI agents

Planning effectively

Give the agent the goal and the specific job, but let it plan the steps. Be specific about what you want, not how to get there.

Action execution

Focus your ask on one page at a time. Agents re-plan when exploring, so complex multi-page tasks should be broken into smaller requests. Make every ask clear and specific.

Data handling

Define the exact structure of output you want before starting. Specify table columns in advance. Provide examples of expected format. Vague requests produce vague results.

Common mistakes to avoid

  • Expecting agents to understand vague requests
  • Asking for multiple pages to be handled simultaneously
  • Not defining the output structure clearly
  • Ignoring ethical and legal boundaries around scraping

Practical implementation

Starting with workflows

  1. Pick one repetitive process you do weekly
  2. Map out the exact steps
  3. Identify what triggers it
  4. Build the workflow in Zapier or Make
  5. Add an AI step for any personalization or judgment
  6. Test thoroughly before relying on it

Starting with agents

  1. Identify a research or data gathering task
  2. Try it manually first to understand the steps
  3. Use Claude for Chrome to automate
  4. Refine your prompts based on results
  5. Scale to more complex tasks once basics work

Key takeaway

The progression matters: Start with workflows for predictable processes. Add MCP for intelligent queries against your data. Use agents for web-based research and automation. Each tool has its place, and using the wrong one creates frustration. Match the tool to the task.

Workshop: Automation Mapping

Identify your automation opportunities and create a simple middleware workflow with at least one AI step.

Deliverables:

  • List of 5+ processes you want to automate
  • Each classified as agent, workflow, or MCP appropriate
  • One designed middleware workflow with AI integration
  • Experience with at least one browser automation tool

Resources

  • Video
    Browser Automation Demo

    Walkthrough of Claude for Chrome and Browse AI for web scraping and research

  • Tutorial
    Middleware Workflow Tutorial

    Step-by-step guide to building your first automated workflow with AI steps