Sending Messages to Workers

I am new to Erlang and using mailing lists, so please forgive my poor
etiquette, and feel free to correct me.

Background:
I am prototyping a simple application using Erlang.  The application has a
manager that reads messages from a socket (UDP in this case) and then sends
the message (based on a destination byte) to a handler process.  There is a
possibility of 2^n handler processes. The number is determined dynamically
at run time based on user input.

A handler process is created when the manager process receives a message to
create the handler. create/3 is part of the handler module's API and is
essentially a wrapper around handler_sup:start_child/2.  When the manager
process receives a packet, it sends a message containing the packet to the
appropriate handler process.  The manager process uses an ets table for
mapping destinations to handler pids.

I implemented the handlers as gen_servers and created a handler_sup that
will restart the handler processes whenever they go down.

Problem:
If a handler process goes down, the destination/pid mapping in the manager
module becomes invalid.  The handler supervisor will generate a new
process, but the manager doesn't know the pid for the replacement process.

Question:
What is the proper way to design this in Erlang?  Do I put the
destination/pid mapping in the handler_sup module and query that whenever
the manager wants to send the message? Or does that violate supervisors
only being supervisors paradigm?

Thanks in advance,

Joseph