I've spent the past several weeks creating and tuning an Autonomous Curation Daemon; better known as a "voting bot." Over the past couple weeks, I noticed that some of my biggest curation rewards are for posts that I never would have voted for manually. Here are several articles that it has voted for (note: I do not speak Korean):
@twinbraid/7guiuv-01
@sochul/extreme-trial-competition-october-23-2016-part-3
@twinbraid/08
@rudxor8/20161020
@yoonjang0707/2016-10-09-10-14
@sanghkaang/3-part-1
Notice anything about this? All of those posts are in Korean. Let me reiterate: I know exactly 0 Korean words. Why did my bot vote for them? Only one conclusion is possible:
By Sirrob01 [CC0], via Wikimedia Commons
My bot speaks Korean.
How is this possible, given that I don't speak Korean?
Well, my bot is different from many of the other voting bots on Steem. Many voting bots have an algorithm that works like this:
if author_reuptation > 50 AND time_since_posting > 27 AND num_votes > 10
THEN vote
This is simple, it's efficient, it's easy to program, and fundamentally there's nothing wrong with it. But my bot is different. I am not disclosing the details of my algorithm yet, but essentially my bot looks at each new post on Steem and predicts how much that post will earn. If the prediction is high enough (and a few other conditions are met, like the author is not on my blacklist and @cheetah hasn't commented on it), my bot votes for the post. Nothing surprising there.
The magic part is what comes next: after the post pays out, the bot goes back and checks to see if its original prediction was good. If the prediction was too high, the bot remembers this; in the future, it is less likely to vote for posts that look like this one. On the other hand, if the prediction was too low (i.e., the post outperformed the bot's expectations), the bot remembers this and makes sure to vote more often for posts like this in the future.
It's a type of learning algorithm called unsupervised reinforcement learning. The "reinforcement" part means that it keeps track of what it's done in the past, and does more of the things that helped and less of the things that hurt. The "unsupervised" part means that I didn't tell it what to look for, I just told it how to look. It learned what to look for all on its own.
Put these two together, and you have a bot that can "learn Korean." If certain aspects of Korean posts have paid out well in the past, my bot will vote for more posts with those aspects - and in so doing, perhaps successfully vote for some Korean posts. Of course, my bot doesn't actually speak Korean. It just knows how to find successful posts...
Credit where credit's due: my bot is written in Python 3.5 and makes liberal use of @xeroc's piston.steem framework, and wouldn't be functional without @furion's steemtools library. Thanks guys! You're both awesome!