November 20, 2022

BSD TCP/IP for Kyu - connect

I now have bind, accept, write, and close working. I am now working on connect and having some troubles.

I see a SYN packet being sent (via Wireshark), but no response. I try first with port 13 (I have a daytime server running), then against port 2345 (I have a server of my own running). In both cases Wireshark shows me the SYN packet being received, but no response. My program that listens on port 2345 never sees anything.

It is as though linux is dropping the SYN packet. I use "service iptables stop" to shut down the firewall, but this yields no benefit.

Normally wireshark does not verify checksums, but it is possible to tell it to do so. And I do. Once I do that, wireshark yields confusing information regarding checksums. There is an IP header checksum as well as a TCP checksum. I never see a bad IP header checksum.

When I make a connection from trona (my linux machine) to "levi" (my kyu test machine). Wireshark claims that the TCP checksums in all the packets that originate on linux are bad. This seems suspicious and unlikely give that the tcp_input routine in Kyu validates these checksums and finds them to be OK. All the TCP response packets sent from Kyu to respond to this connection are indicated by Wireshark to have valid checksums. And everything works.

When I watch an attempt to connect via wireshark, I see a SYN packet, and it gets repeated (retrasmitted) after 5 and then 24 seconds. Wireshark claims the checksum is bad, but I don't know whether to believe it.

Now I run wireshark on "tcp port 2345" and use "telnet trona 2345" to test my server. I have to change the capture to localhost, but then I see all the traffic. Wireshark claims that each and every TCP checksum is invalid (coming and going) yet everything works.

The SYN packet is 74 bytes in size. The only flag set is SYN. TCP header length is 40 bytes, along with a 20 byte IP header, and a bogus 14 bytes ethernet header to make 74 bytes. Source port is 42504. There are 20 bytes of options:

mss 65495 bytes
sack permitted
timestamps (8 bytes).
window scale 7 (multiply by 128)

Watching interface eno1 using "ether host c2....5e" I see the following. The SYN packet is 74 bytes once again. The IP header checksum is good. Only the SYN flag is set. The source port is 1024 (odd?). Again there are 20 bytes of options:

mss 512
timestamps (8 bytes)
window scale 0 (multiply by 1)
Nothing about SACK being permitted. The mss value seems awfully small.

Try port 2345 from my windows machine

I can use "telnet trona 2345" from a Command prompt window and it works. Looking at the wireshark capture, the initial SYN packet is indicated as having a good checksum. Again, all of the responses from trona are flagged by wireshark as having a bad checksum (yet everything works). Actually the last response from trona (the ACK to the FIN/ACK) is shown as having a good checksum.

So my conclusion is that Wireshark is just goofy about the checksums. It claims that many (but not all) checksums on packets coming from the linux machine (the same one on which wireshark is running) are bad. So I think we have to ignore what wireshark says about those packets and figure that the linux TCP/IP code is actually OK, which is reasonable. Wireshark does approve of all the packets from Kyu, once things are working, so I think we can use wireshark to diagnose my Kyu TCP efforts.

Get it all to work

I spent an absurd amount of time writing some code to display packets, mbufs, and to validate checksums. I learned that the BSD code does some crazy things to calculate TCP checksums, and all bets are off about what the "len" field in the IP header means at various points in time. I did manage to sort all this out and learn about things I never wanted to learn about.

I moved some code from tcp_output() into tcp_template() that does a funky hack to install our IP address into the source IP field properly. This really belongs back in a netif structure and probably should be handled in pcb_connect(), but at this point things work and I am satisfied for now. I'll no doubt rediscover this hack someday in the future and hate myself.

I tried connecting to various ports on my linux machine from Kyu. I usually get an RST response (as I should) when no server is running. I got surprised by port 111. I got a nice friendly FIN response as though there was actually a server running. I just picked 111 at random. It turns out that part of Sun NFS (the ONC RPC portmapper service) is running on port 111. Finally I connect to port 13 where I have the daytime service running, and everthing looks good. Data even got transferred, but I don't yet have any way to read what must be waiting in a socketbuf.


Have any comments? Questions? Drop me a line!

Kyu / [email protected]