well, it somwhat depends on how your router works,
But for most off the shelf routers, they use something called "NAT" network address translation. There is no need to configure a proxy server.
The router will have a private IP Address that it will assign to your machine with DHCP which will...