Age | Commit message (Collapse) | Author |
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
netif_rx queues things up to a per-cpu backlog, whereas
netif_receive_skb immediately delivers the packet to the underlying
network device and mostly never fails. In the event where decrypting
packets is actually happening faster than the networking subsystem
receive them -- like with 65k packets with UDPv6 in `make test-qemu`
-- then this backlog fills up and we wind up dropping some packets.
This is fine and not all together terrible, but it does raise the
question of why we bothered spending CPU cycles decrypting those
packets if they were just going to be dropped anyway. So, moving from
netif_rx to netif_receive_skb means that whatever time netif_receive_skb
needs winds up slowing down the dequeuing of decryption packets, which
in turn means the decryption receive queue fills up sooner, so that we
drop packets before decryption, rather than after, thus saving precious
CPU cycles.
Potential downsides of this include not keeping the cache hot, or not
inundating the network subsystem with as many packets per second as
possible, but in preliminary benchmarks, no difference has yet been
observed.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
This removes our dependency on padata and moves to a different mode of
multiprocessing that is more efficient.
This began as Samuel Holland's GSoC project and was gradually
reworked/redesigned/rebased into this present commit, which is a
combination of his initial contribution and my subsequent rewriting and
redesigning.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Suggested-by: Mathias Hall-Andersen <mathias@hall-andersen.dk>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Suggested-by: Mathias Hall-Andersen <mathias@hall-andersen.dk>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Suggested-by: Mathias Hall-Andersen <mathias@hall-andersen.dk>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
DaveM prefers it to be this way per [1].
[1] http://www.spinics.net/lists/netdev/msg443992.html
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
This caused certain packets to be rejected that shouldn't be rejected,
in the case of certain scatter-gather ethernet drivers doing GRO pulling
right up to the UDP bounds but not beyond. This caused certain TCP
connections to fail.
Thanks very much to Reuben for providing access to the machine to debug
this regression.
Reported-by: Reuben Martin <reuben.m@gmail.com>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
This way is more correct and ensures we're within the skb head.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
All locks are potentially between user context and softirq,
which means we need to take the _bh variant.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
If it's time to rekey, and the responder sends a message, the initator
will begin the rekeying when sending his response message. In the worst
case, this response message will actually just be the keepalive. This
generally works well, with the one edge case of the message arriving
less than 10 seconds before key expiration, in which the keepalive is
not sufficient. In this case, we simply rehandshake immediately.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
With the prior behavior, when sending a packet, we checked to see if it
was about time to start a new handshake, and if we were past a certain
time, we started it. For the responder, we made that time a bit further
in the future than for the initiator, to prevent the thundering herd
problem of them both starting at the same time. However, this was
flawed.
If both parties stopped communicating after 2.2 minutes, and then one
party decided to initiate a TCP connection before the 3 minute mark, the
currently open session would be used. However, because it was after the
2.2 minute mark, both peers would try to initiate a handshake upon
sending their first packet. The errant flow was as follows:
1. Peer A sends SYN.
2. Peer A sees that his key is getting old and initiates new handshake.
3. Peer B receives SYN and sends ACK.
4. Peer B sees that his key is getting old and initiates new handshake.
Since these events happened after the 2.2 minute mark, there's no delay
between handshake initiations, and problems begin. The new behavior is
changed to:
1. Peer A sends SYN.
2. Peer A sees that his key is getting old and initiates new handshake.
3. Peer B receives SYN and sends ACK.
4. Peer B sees that his key is getting old and schedules a delayed
handshake for 12.5 seconds in the future.
5. Peer B receives handshake initiation and cancels scheduled handshake.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
We're not leaking the DSCP, but we do deal with ECN.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
It should never be the case that skb->head + skb->transport_header -
skb->data is greater than 2^16, but in case the kernel network stack
borks this at some point in the future, we don't want this to slyly
introduce a vulnerability into WireGuard.
Further, really smart compilers might be able to make deductions about
data_offset, and optimize accordingly.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|