THIS IS MY
Card Sorter Project
About
PROJECT PURPOSE
This project’s purpose is to design and create a device that can sort a set of Magic the Gathering trading cards by:
-
Using image recognition to match each card captured with a camera to its information collected from an online database
-
Using a robotic arm to move only one card at a time without damaging it
-
Separating cards into different piles based on parameters selected by the user
​
​
Initial Hardware Design
-
A wooden frame contains all the components listed in the next slide
-
One half of the frame contains the robotic arm when fully contracted
-
Another half is divided into four bins accessible by the arm that each hold a pile of cards
Frame Construction
CONSTRUCTING THE WOODEN FRAME TO HOUSE ALL OTHER COMPONENTS
As outlined in the sketch of the initial hardware design slide above, I created a wooden frame to house all of the other components. I used a table saw to cut plywood into interlocking sheets and then glued them together, leaving one side unglued and instead held in place with a screw.​
Within this basic wooden frame, I installed a stationary and adjustable camera mount due to connector limitations, drawer slides to support the arm, a stepper motor and screw rail shaft to drive it horizontally, a limit switch, a power supply plug, and switches to control separate power supplied to the Pi and motor.
​
​
Arm and Electronics
CONSTRUCTING THE ARM AND ELECTRONICS
Over a handful of different iterations, I made a number of improvements to the initial hardware design of the arm. As you can see below, a single servo motor is used to drive the suction cup that picks up each card vertically while a mini pump was added to create a vacuum seal to reliably lift each individual card. Additionally, a limit sensor was added to ensure that the servo motor doesn't overextend the arm downward when picking up cards from a significantly tall stack.
​
Two high amperage voltage regulators have been added to provide 3.3V to the stepper motor and 5V to the Raspberry Pi and other components.
​
Lastly, the Raspberry Pi and touch screen were fitted on top, and the vast majority of electronics were added inside this frame with a prototype shield including a motor controller and a number of MOSFETs used to step up the 3.3V pinout signals to a useable 5V signal.
​
FLANN Based Matching
FIRST ATTEMPT AT IMAGE RECOGNITION
My first attempt at implementing image recognition was to create a FLANN (Fast Library for Approximate Nearest Neighbors) based matcher using OpenCV. This matcher would match nodes, prominent angles within an image, from an image captured by the Raspberry Pi's camera with nodes it has been trained with from the MTG library of card images.
​
As you can see below, this proved to function quite well when recognizing a single card with a matcher trained with that one card. However, as the matcher was trained with more cards, the process became more inaccurate and began taking an unreasonable amount of time as it overburdened the Pi's limited processing power.
​
​
Contour Detection and Difference Hashing
SECOND ATTEMPT AT IMAGE RECOGNITION
My second attempt at implementing image recognition was to create a library of image hashes, hash values that uniquely identify an image based on its contents, using OpenCV and the art of each card extracted from the MTG library of card images. Then using contour detection, I was able to consistently find either the border inside the black outermost border or the border around outside of the card art from the images captured using the Pi's camera and by using difference hashing, calculating the hamming difference between two hashes to find the degree of similarity, I was able to find the closest match from this database.
​
As you can see below, this accurately identified cards from a complete database of over 20,000 unique MTG cards, doing so quickly and reliably despite hardware limitations. The only limitation to this solution is its inability to identify non-rectangular card art borders. However, these only exist on Unhinged cards, joke cards that are of little to no significance to most players, and so I chose to move forward with this solution.
​
​
Software Development
IMPLEMENTING A SIMPLE GUI AND SORTING ALGORITHM
With the image recognition working and the device itself assembled, the next step was to develop the software to put it all together.
​
First, I created card_sorter_main.py to serve as the starting point for the program and the main app that all other Qt4 windows belong to.
​
Then, I created card_database.py which uses the Magic: The Gathering SDK to download a list of every MTG card, download each image, and add the image hash of each card's artwork to a dictionary. A simple query can then compare any scanned card's image hash to find an appropriate match and look up all of the card's data. Because of the massive amount of data that must be downloaded, I also included fail safes to ensure that if the download process was interrupted or more cards were added to the MTG library itself, the database could continue downloading from where it left off without having to redownload every card from the beginning.
​
Next, I created the GUI: sorting_window.py serves as the front end, providing the user with options to sort cards by card type, test_window.py provides full manual control of the arm through arm.py for debugging mechanical issues, and scanner_window.py allows the user to test the image recognition through scanner.py and to set the empty image used to determine when the first bin is empty.
​
I decided to keep the sorting options simple and only offer sorting cards by type as a basic proof of concept. You can see the entire commented program in the link to its GitHub repository below.
Demonstration and Conclusion
DEMONSTRATING THE FINAL PRODUCT AND SUMMARIZING HOW IT COULD BE APPLIED IN SOCIETY AND WHAT I'VE LEARNED MAKING IT
Below is a demonstration of the finished device. After selecting "Artifact" for the first bin and "Land" for the second, the device successfully identified each card in the leftmost bin before moving it to one of the middle bins if it was an artifact or land card or moving it to the rightmost bin if it was neither card type.
​
With further improvements to its software and speed, such a device could be marketed to both card collectors and shops looking to reorganize their collections and even keep a digital record to quickly and easily find any card in their entire collection.
​
However, simply working this project has taught me about a number of things including the Python language, OpenCV and image and video recognition as a whole, the application of mechatronics, and time and resource management.
​
​