Re: Robocode and reinforcement learning

In my first attempt to integrate Robocode with my RL server, I tried to
> implement a REST API for the RL server. In this setup, the robot made
> REST/PUT requests to send game state to the RL server and retrieve actions
> to execute. It was possible to do this in a single REST/PUT command; but it
> was too slow (REST overhead).
>
Interesting, I would not think that REST would make enough difference.

We already have internal/custom binary protocol for that:
actions == ExecCommands
observations == ExecResults
https://github.com/robo-code/robocode/blob/master/robocode.core/src/main/java/net/sf/robocode/peer/ExecCommands.java#L294
https://github.com/robo-code/robocode/blob/master/robocode.core/src/main/java/net/sf/robocode/peer/ExecResults.java#L86

I implemented it years ago when I wanted to run a .NET robot and I did some
in-process IPC.

Is that good enough or we should prefer something more formal ?
Like gRPC or protocol buffers in order to make it public contract/API ?


> By design, the Robocode robot does not send a reward value to the RL
> server. This allows the RL server to specify its own reward function. The
> reward function is a fundamental (and interesting) aspect of RL agent
> design, and I wanted to provide full flexibility to the RL agent designer.
>
Definitely your RL needs to re-model the rewards. I'm just talking about
Robocode's natural score. It evolves with each turn.
Maybe that's philosophically part of RL `observations` ?


> I'm not so sure. Perhaps I'm missing something specific to Robocode, but
> it seems like the community might be able to host their own remote Robot
> implementations (i.e., servers as I've described above).
>
Unless every author with the remote robot is willing to provide docker
image, sharing my-robot-how-to-host-it on diverse platforms would not fly.
That's why I'm suggesting that any robot author who wants to have his robot
in the competition would have his server open and available to incoming
battles all the time. On authors own servers.

-- 
You received this message because you are subscribed to the Google Groups "robocode" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion on the web visit https://groups.google.com/d/msgid/robocode/CABB%2B072UvY5PLuXJTFV7rk61ym1gEKP1zgC1O2kh5iuhxRy%3Dnw%40mail.gmail.com.