peer to peer UDP communication across the LAN .

Question

I am trying to transfer audio /video  packets from one LAN (private IP) to another LAN but not able to receive it in peer to peer communication using UDP protocol.

Whereas the same application is working well inside the LAN and also when connection is in between one LAN to public IP(viceversa) and able to see the video on both ends.

I am able to communicate using server but not able to send packets using peer to peer.
I have gone through so many articles in Internet there I found it can be NAT or UDP port problem.

When I am calling a user as per the available status info. I am sending my own local IP address and using particular port.Other end user is able top get the incoming message through server code but not able to receive the packets.I have tried all permutations but not able to catch the exact problem.

Any pointers or link ,suggestions......... can focus light on the problem.
Thank you.