How to Build a Game Bot using Template Matching in Python

Introduction:

In this guide, you’ll learn how to build a basic game bot using Template Matching in Python. Template Matching is a technique in computer vision used to find parts of an image that match a template image. This approach is particularly useful for creating bots that can interact with games by identifying key elements such as buttons, icons, or specific game characters.

Let’s dive into the step-by-step process of creating a simple game bot using Python’s OpenCV library.

Step 1: Install the Required Libraries

To get started, you need Python installed on your machine along with some essential libraries.

Run the following commands to install the libraries:

pip install opencv-python pyautogui numpy

OpenCV: Used for image processing and template matching.
PyAutoGUI: Allows for mouse and keyboard control, simulating bot actions.
NumPy: Used for efficient array processing.

Step 2: Capture Game Screenshots

Before you can perform Template Matching, you need screenshots from the game. You can take them manually or use Python to capture them.

Here’s how to capture a screenshot using PyAutoGUI:

import pyautogui

# Capture a screenshot and save it
screenshot = pyautogui.screenshot()
screenshot.save('game_screenshot.png')

You can also use external tools like Snipping Tool or screenshot functions in your OS.

Step 3: Create Template Images

The bot needs reference images, called templates, to match against the game screen. These could be images of buttons, characters, or specific game elements you want the bot to interact with.

Make sure the template images are cropped properly and saved with high quality. Store them in a folder for easy access.

Step 4: Template Matching Logic

Now, let’s write the core logic for Template Matching using OpenCV.

Read the screenshot: First, you’ll load the screenshot of the game.
Read the template: Then, load the template image you’re trying to match.
Apply Template Matching: Use OpenCV’s cv2.matchTemplate to find matches.

Here’s a basic example:

import cv2
import numpy as np

# Load the screenshot and template image
screenshot = cv2.imread('game_screenshot.png')
template = cv2.imread('button_template.png', 0)

# Convert screenshot to grayscale for matching
screenshot_gray = cv2.cvtColor(screenshot, cv2.COLOR_BGR2GRAY)

# Perform template matching
result = cv2.matchTemplate(screenshot_gray, template, cv2.TM_CCOEFF_NORMED)

# Set a threshold for matching accuracy
threshold = 0.8
locations = np.where(result >= threshold)

# Draw rectangles around matched regions
for pt in zip(*locations[::-1]):
    cv2.rectangle(screenshot, pt, (pt[0] + template.shape[1], pt[1] + template.shape[0]), (0, 255, 0), 2)

# Display the result
cv2.imshow('Detected Matches', screenshot)
cv2.waitKey(0)
cv2.destroyAllWindows()

In this code:

cv2.matchTemplate compares the template with the screenshot.
np.where finds the locations where the match accuracy exceeds the threshold.
cv2.rectangle draws a green rectangle around the detected matches.

Step 5: Simulate Mouse Actions

Once the template is detected, you can use PyAutoGUI to simulate mouse clicks.

Here’s how you can move the mouse to the matched template location and click:

import pyautogui

# Get the first matching point
for pt in zip(*locations[::-1]):
    pyautogui.moveTo(pt[0] + template.shape[1] // 2, pt[1] + template.shape[0] // 2)
    pyautogui.click()
    break  # Remove if you want to click all matches

In this snippet, the mouse is moved to the center of the matched area, and then a click is simulated.

Step 6: Automate Game Actions

Now, you can chain multiple steps by detecting different game elements (like attack buttons, restart buttons, or character positions) and writing a loop to automate the bot’s behavior.

Here’s an example of detecting a “Play” button, clicking it, and then waiting for a new screen to load:

import time

while True:
    # Take a new screenshot each iteration
    screenshot = pyautogui.screenshot()
    screenshot.save('current_screenshot.png')

    # Load the updated screenshot
    screenshot_gray = cv2.cvtColor(cv2.imread('current_screenshot.png'), cv2.COLOR_BGR2GRAY)

    # Match the play button template
    result = cv2.matchTemplate(screenshot_gray, template, cv2.TM_CCOEFF_NORMED)
    locations = np.where(result >= threshold)

    if len(locations[0]) > 0:
        # Click the play button if found
        for pt in zip(*locations[::-1]):
            pyautogui.moveTo(pt[0] + template.shape[1] // 2, pt[1] + template.shape[0] // 2)
            pyautogui.click()
            break

    # Wait before taking the next screenshot
    time.sleep(2)

Step 7: Adding Robustness

To make your bot more reliable, consider:

Multiple Resolutions: Game elements may change sizes depending on the resolution, so create multiple template images or dynamically resize them.
Error Handling: Use try-except blocks to handle unexpected game states.
Delays and Pauses: Introduce delays using time.sleep() between actions to mimic human behavior and avoid being detected as a bot.

Step 8: Testing and Refinement

Now, test your bot in the game. You may need to tweak:

The template matching accuracy threshold.
The size and quality of your template images.
The timing and sequence of actions in the game loop.

Conclusion

Building a game bot using Template Matching in Python involves several steps, from capturing screenshots to detecting elements on the screen and simulating clicks. With OpenCV and PyAutoGUI, you can create a basic game bot that performs automated tasks in a game.

Next Steps:

Make the bot more flexible to work on different games with minimal changes.

Enhance the bot’s intelligence using advanced techniques like object detection.

SeeB4Coding