Final Plenary session
Friday 19 October 2018
11 a.m.
BRIAN NISBET: Good morning, and welcome to the final Plenary session of RIPE 77. It is still RIPE 77, the week has been long, it could be RIPE 78 already.
So, before the PC hands over control of the meeting back to Hans Petter, we have a last few talks for you all to excite you and enervate you on your trip home. I have a presentation now on RPKI and some lightning talks and then we'll hand back to Hans Petter to finish up the meeting.
But just first, the results of the PC election which I know you have all been eagerly waiting for. Thank you to all eight people who volunteered, it was great to see that many people and some new voices, some people who have been around a while longer, but great to see and something I hope that we continue to see.
All of the numbers will be published, but the two people who were successful in election were Maria Isabel Gandia and Alireza Vaziri. So thank you very much.
(Applause)
And again, thank you to all people who put themselves forward. There'll be another opportunity to do all of this in sixth months' time in recognise Nick and I would remind you all you can still rate the talks for this morning as well and please do give us the feedback so we can continue to produce the Plenary content that you want because that's really what's important.
Without further ado, I'll ask Louis from CloudFlare to talk to RPKI.
LOUIS PONSIGNON: Good morning. It's a pleasure to be here. Thanks to RIPE. I'm going to present about RPKI this morning. Because, we recently rolled out RPKI at CloudFlare, so we tried to do a scaled deployment. So, just a quick introduction what is CloudFlare.
We are a CDN. We cache files, we protect against attacks, we have more than 150 points of prints. We present in more than 72 countries and 186 Internet Exchange. We serve approximately 10% of all RIPE requests. We also do DNS, a lot of queries.
And who am I? I am a network engineer. I also work a lot on data. And I work at CloudFlare. I used to work at CloudFlare London, now I am in San Francisco and I built some Open Source tools so I invite you to check that out to do data collection.
Let's look about RPKI.
So just a brief introduction to remind people if you don't know what and how RPKI works.
So there are two RFCs that define how it works. The first one says to cryptographically sign announcements routes, ASN and length.
The second RFC communicates the prefix to a router.
So, all the RIRs have a certificate that you decide to trust, and the RIRs are going to generate an organisation certificate. And with that, you can create route announcements. You can create ‑‑ I own this prefix so that means I'm going to sign it and I'm going to sign it. I know it can be announced with an ASN. Usually you are going to put your ASN.
How validation works, you are going to have a validator which are going to gather all the certificates and all the ROAs which are the route recalls and cryptographically check them so that means they are going to discard invalid signatures and only do a prefix list of the valid route announcements. Once this is done, it forwards a list to a cache, like a RTR cache, which then is going to communicate over the RTR protocol to a router. And then, in summary, the router, when it receives an announcement, for instance, here, when the hijacking router announces more specific, the ISP router, when you are going to send a packet to a destination that's been hijacked, you are going to send it to the bad router, but with RPKI, it's going to compare, it's going to check that the route is actually authorised and in this case it's not, it's not allowed to be a /24, it's only to be allowed to be a /23. So the route is going to be discarded and the route is going to be sent to the correction destination.
So use case, mostly filter bad announcements, either mistakes or hijacks and you can also use it as in your own IP service to say make sure that your clients are actually sure of the range.
Just a quick cover in the BGP leak and hijack.
Why do you have to sign? Well, hijacks happen. And recently, like back in April, a DNS resolver were affected. So, for instance, somebody in the Chicago area sent a bad announcement for Amazon DNS ‑‑ like the IP that are used for Amazon. And when the resolver was actually querying Amazon for a Bitcoin website, it actually queried the hijacked prefixes and then the hijacked prefix ‑‑ the hijacked server ‑‑ the servers who are actually managing the hijacked IPs were returning fake results, which led to this.
And when the user was actually fetching the website, he had had a certificate error. Some of them actually clicked through it and entered credentials which allowed the hijacker to get credentials.
The availability is like rendering, why would you do a hijack? You can make a resource unreachable. Or impersonating, which is integrating confidentiality. So the protocol trace the other ones like on the upper layers who don't protect. So for instance, DNSSEC or HTTPS, those ones actually ensure that you are talking to a trusted authority. If you don't have that, for instance, on UDP, it won't ‑‑ you can ‑‑ it will ‑‑ you will be ‑‑ you can hijack and send bad response.
For instance, a few cases of hijacks can happen. The first one is you just send the route with your ASN. It may become the shortest AS path in some places. But BGP origin validation in RPKI could filter it out. It's sensitive on IX because shortest path.
In the second case the hijacking party is impersonating, like faking the origin ASN. So you get ‑‑ it may ‑‑ RPKI ‑‑ BGP origin validation is out of the scope of the second one, but it's less sensitive to hijacks because the AS is going to be longer, thus the BGP preference will be lower. It's the sensitive on IXs.
And in the last case you are announcing a more specific IPv6. In this one, RPKI filter it out if you sign it correctly. But, it will be preferred in any case because more specific.
So, we have very localised attack and while waiting on RPKI, what are the other alternatives? Like you have IRR filtering. You are not sure that the generated announcement is actually valid, like, you are not sure that the actual owner of the prefix wrote it. RPKI enables this.
And there is more or less some people do it for critical resources like DNS announcing the more specific which avoids the last, the third case, you avoid the more specific attacks but it's not very advised to do the second point, it's more like a patch up, because you increase the size of the BGP table.
So, what does 'at scale' mean? How did we deploy at CloudFlare, how did we deploy at scale? So, we have addresses in all 5 regions. And we wanted to make prefix signing in validation and make sure that, on the long term, once, like, in ten years, that actually the person who is going to be in charge of RPKI at CloudFlare is still going to be able to sign this. And we want to enable strict validation at scale as well and monitor everything.
So, we had to choose between like hosted or delegated. Hosted means the ROA actually manges the RIR creation and everything is maintained by the ROA. Delegated means we actually manage a certificate authority, we manage all the ROAs. We realised that some ROAs do not allow to do delegated, for instance RIPE and LACNIC. But we chose hosted because we do not allocate IP addresses, we do very few changes, we don't need somebody to ‑‑ we don't want to delegate to somebody else to sign this.
And there was only a handful of software for maintaining a CA, and you have to maintain Rsync server, so, we decided to go with hosted.
And it doesn't just ‑‑ just the last point, in the failure scenario, it doesn't change, hosted or delegated doesn't change. If the RIR certificate is hijacked it is compromised in both scenario, like, it's still a failure.
But what we mostly want is APIs, so just inserting, updating, listing and deleting. So in APNIC, apparently it's a draft. AFRINIC uses the APNIC version. ARIN uses its own version of RPKI ‑‑ I mean, its own version, its own RPKI, which openly has inserting. Cannot list, cannot update, cannot delete, other than connecting to the website. For the RIPE and LACNIC, you have no API, but it's easier to patch.
Regarding availability, since we want to do validation, we want to be sure that we can fetch the CA. So, they are in the five region they are not very distributed, so we just tried this with RIPE Atlas just targeting the RPKIs, and we realised that they are very far. In four out of five cases, you are very far away from the RPKI. Even in east Asia, actually there is GPNIC, like, it doesn't allow much caching and we need to support scale. So from Sydney, for instance, how long does it take to synchronise the actual RPKI? So the RIPE RPKI, 90 megabyte took 5 minutes, because there is a huge latency between Sydney and Amsterdam.
So, in the future ‑ like, right now, there is a bit more than 10% of route sign, and if everything was signed, we probably would have 1 gigabyte to download at 2 to 4 megabits per second, we take 30 minutes to one hour just to synchronise certificates. It's painful. And the database, you can actually launch an attack, you can fill your database with random records. I am just pointing at this one which actually is a legitimate one that you can find. But, for instance, this one, I'm not going to use it in any case, I'm not going to ‑‑ it's a /64, it's never going to be announced on the Internet.
So, we have 150 PoPs and how we are going to do validation in every single one of them.
We could do RTR from a central point to each router. But, that's a single point of failure, there is latency, which site are we picking? Do we pick the US, do we pick Europe? And what about all PoPs in Australia?
But the major drawback of this is, there is no encryption. Because, when RTR was created, there is multiple mode, there is TLS, TCPAO, but vendors mostly implemented over TCP, which is very insecure, you can break the connection, you can hijack it, you can insert bad prefixes when you communicate the prefix to the router, you can create bad prefixes.
We could do a validator on every POP, but it's kind of a waste of resources like maintaining, fetching everywhere and there is still the latency problem. I'm not talking about North America or Europe, it's going to be fast to load the RPKI. But in some remote sites it may be ‑‑ it may take a while.
It's harder to monitor because you have to gather all the statistics back.
So we had kind of an in‑between solution. Let's have a local cache in every point of presence and use the CDN to have the secure prefix list. So central validation, which is in our network, have authority of the prefixes and the custom RTR software communicates with router are locally installed. We have local RTR software which fetches the content. It was easy because it's a very small tool that we integrated with Salt and all the pipelines, automatically built and deployed and automatically started.
So, how does it look? Like this.
The validator is going to fetch all the RPKI, going to create a prefix list that we are going to display it in JSON behind our CDN and then go RTR is going to fetch, is going to fetch this prefix list. And then communicate with the router over a secure link. I mean it's local, so we want to ‑‑ it's more ‑‑ I'm going to say it's more secure than being on the Internet. It's going to be very close to the router.
So, you can fetch all a list of prefixes here and you can fetch the Open Source implementation on the second link.
It's just a very small software that we built that do TCP and TLS to send routes. We tested it. Works with Juniper.
And soon, what we hope to do is like if you want to just test RPKI implementation, we want to do a service behind CloudFlare spectrum which is a proxy. So you have got nothing to install you just enter CloudFlare IP and you get the prefix list.
How about monitoring? So, we want to monitor the RPKI. So this is the great job from the Crypto team, is they started the CloudFlare certificate transparency, and they also put something for RPKI. So that means we trust the certificates and then we send all the certificates to the certificate transparency, which means there is a history you can trace back to every certificate that was emitted by the RIR. You can see the revocation, you can see everyone that was accepted. It doesn't contain the ROAs. It just contains certificates which means every organisation.
Monitoring of validation. Internally, we have some dashboards. Coming from a validator, how many ROAs, and from which ROA, what's the distribution? Coming from our edge, how many invalids and villed, how many filtered and valid routes? And online, just like we use a lot of services, we should check all that is correct. So, for instance, on our dashboard we see how many CloudFlare routes are actually valid, that means how many routes we sign. We sign approximately 25% of our routes. No invalid, so so far so good. And we want to do some monitoring of filtering, which is a bit harder. It's a long‑term project for this one, but due to our presence in 180 IX, we want to announce a prefix, a very specific prefix, /24, for instance, and make it invalid and of the enclosing prefix. So that means that if you do RPKI validation, you are not going to accept the more specific, thus your packets are going to be routed to the less specific. And this is how we want to test if the progression of RPKI filtering, if you implemented it, if the IX implemented it.
And here it is. Do you have any questions?
AUDIENCE SPEAKER: Hi, Rumy from RIPE NCC. I have a question on remote chat from Jason as a person: What percentage of your routers are currently doing strict RPKI‑based route origin at validation, do you have a rough timeline when it will be at 100%.
LOUIS PONSIGNON: Now, right now, two. We are continually deploying more. It's planned for the end of the year that all our major sites are doing RPKI ‑‑ BGP origin validation. We have ‑‑ YES, mostly Junipers, we use mostly Junipers so those ones ‑‑ we know they are compatible. We have a few others vendors, which we need to test before. But it's ‑‑ before the end of the year, we hope to have deployed BGP origin validation everywhere.
AUDIENCE SPEAKER: Tim Bruijnzeels. Just two small corrections if I may on the CA side of things. Because you said that it's impossible to run your sown CA below the RIPE NCC infrastructure, and while it's not widely advertised, that's actually not true, it is possible to do that.
Secondly, there is also an API available there that you can use, whether you want to use five different APIs to achieve your goals that's a completely different question but it is there.
More importantly, maybe at this point, about validation, I really like how you set this up for your infrastructure, I think it makes a lot of sense to me that you would scale that way. But I have some doubts about making your JSON available to people and basically asking people to outsource their validation function to you. I mean how would they know what you do with regard to local exceptions etc.? Also, and this is coming back from feedback that I have heard from people that started adopting invalids here, sometimes they get people call them and sometimes they make an exception for sometime and then the other party usually fixes the issue and they carry on, and that would be very difficult to do with the feed that they get from you, for example.
LOUIS PONSIGNON: To your first comment, I'll check again for the RIPE. For your second thing, why would they trust CloudFlare? We made that mostly to help deployment of RPKI. If you choose not to trust us, you are free to also run your own validator. In the same way we have a DNS resolver, you choose to use us or you can run your own DNS. We want also to do that to provide this list, because we want to know which prefix we are filtering on. So, you can see, since it's public, you can see which ones are filtered. Feel free to use it. And I'm sorry, I forgot the last question.
BRIAN NISBET: I am sure you can talk afterwards.
AUDIENCE SPEAKER: Another question from Jason as a person.
Are you measuring how much traffic you are currently getting from RPKI invalid IP address space and, if so, can you compare some rough figures, zero traffic, very few traffic or substantial amount of traffic?
LOUIS PONSIGNON: I didn't count exactly, but from the small tests, I haven't tried recently, to not a huge amount of traffic, but I need to check again, so I don't want to give any numbers yet.
AUDIENCE SPEAKER: And a follow‑up comment from the same person. I support Tim's statement about outsourcing validation.
RANDY BUSH: If you page back about three meetings or four meetings ago, you'll see my presentation that I run a subsidiary CA under RIPE for a year, 18 months now, and there is a subsidiary CA under me.
LOUIS PONSIGNON: Okay.
AUDIENCE SPEAKER: Hi. Alex Band. With regard to ‑‑ now first of all, big kudos to what you are doing. I mean, embracing this technology and deploying it at this scale, I think it's impressive. But, if I look at GoRTR on GitHub and the way you formulate it on how you can use this tool, it doesn't really imply that this is for testing only. You go this is a JSON feed and this is crated and I don't know what occur ated means and oh by the way here are some alternative feeds which use which don't use hospitals, which are aren't necessarily modern you don't really know what to expect. I think it would be a wise choice to be to be very clear that this is something you can only use for testing and getting some operational experience but should in no way be used as a production source for using route validation. So I would really like to you update the documentation and the expectations that you set on GitHub.
LOUIS PONSIGNON: Yes, if you read it, I list there is no guarantee, I just say we use it in production internally, feel free to use it. We took the same format as a JSON for, as the RIPE validator, we tested it against it, feel free to use the CloudFlare as a source, or any other source, you can even take any JSON file that's on the same RPKI validator format, and it will communicate the prefix to your router. You are not forced to use the CloudFlare prefix list.
AUDIENCE SPEAKER: Andreas Polyrakis. Personally I find the service very useful. Of course I understand the concerns people that say production network should not rely on someone else's validator. But although this is absolutely true for larger networks, smaller networks who do not have the capacity to do it on themselves and in some cases this might be useful for those smaller ones compared to doing nothing. And apart from this, there is also a second use, I think that they also mentioned testing but even having a baseline for comparison for what our own validator service produces, this is also useful because if our validators ran okay, if they produce right results, compared to what someone else produces so this is also a useful thing so it is a useful service after all from my perspective.
AUDIENCE SPEAKER: Tim. I just want to clarify one thing because maybe it wasn't clear about the use of GoRTR. I really understand and like how you do this to make this thing scale your own infrastructure and that may work well for others as well. You have a validator that you maintain locally and you have multiple of these. Your points of presence and it gives you redundancy and all that, I really like that part of it. I was merely making a comment about what source you use for the validation input.
LOUIS PONSIGNON: Internal RPKI from the RIPE.
BRIAN NISBET: Okay. So, there are two people at the mics and that's it.
RUDIGER VOLK: Not wanting to beat the, well, okay, outsourcing security theme to the death. I think that in all cases where there is an offer or a suggestion or a possibility for using a third‑party's evaluation of security, and in particular for RPKI, it would be very good if explicit statement was saying this ‑‑ if as a third‑party you are using this, you should be aware that this is not what the design is ‑‑ what's suggested in the design and you are really on your own. Well, okay, that just a hint the guys at the IXPs who are doing the evaluation for third parties probably should put in that warning as well. The parties that actually use that outsourced security service should be perfectly aware what they are doing, and parties that offer something that can be used this way should explicitly warn about it, but people are in control of their own fate and indeed there may be parties that are running something themselves which actually fail in cases that could be avoided.
AUDIENCE SPEAKER: Another question from Jason. First of all, big thank you for pushing RPKI. And the question is: There is a delta of about 1,500 ROAs if we compare CloudFlare's data‑feed with what a RIPE NCC validator 3 provides. Do you know why there is such a big difference, 60K CloudFlare, 61.5K by a local validator, for instance?
LOUIS PONSIGNON: How many did he say?
BRIAN NISBET: You have 60, he is saying 61.5.
AUDIENCE SPEAKER: 60,000 CloudFlare and 61.5 by a local validator.
LOUIS PONSIGNON: I'll take a look. It's probably because we removed some of them, the ones we are likely not going to filter on, but yes, I will take a look.
BRIAN NISBET: Thank you very much.
(Applause)
And next up we have the every popular NCC technical review of the week from Razvan.
RASVAN OPREA: Hello everyone, good morning. I have been part of the technical team for quite a while now and it's fantastic to see familiar faces, but ever growing number as well of new faces. And for those of you who are new to this meeting, well this is the technical report in which we outline the most important bits of our infrastructure, how we have done it, and we present to the team that has made this possible.
First of all, everywhere we go, also when we're here in Amsterdam, we have a connectivity sponsor, and the job of the connectivity sponsor is to provide us with the uplink that you are going to use, and this time it's been NLix, they provided us with a dedicated fibre which was commissioned by KPN, usually wherever we go we have a dedicated fibre to the venue. Minimum of 1 gigabit. And this time, they have given us 10 gigabit connection. We had transit from ‑‑ (applause) ‑‑ well, you haven't used much of it, to be honest, so... so you need to try harder.
We had transit from Joint Transit and Open Peering and we also had IXP peering and NL‑ix route servers.
Besides the uplink itself, the wire network consists of many other, let's say, network services that we provide the webcast. It's also being done by our team, Terminal Room set‑up, the presentation kit that you see here, and it's done all in a period of three days before the meeting starts, plus, of course, the preparations that we do in the office prior to that.
Since we're at the uplink, this is the traffic that you have done. And you can see it's nowhere near what we have been offered. So, yes, please, next time, try a bit harder to use it. Right.
We have been using Ubiquiti routers. We had, for this meeting, new 10g interface edge router infinities, but we had some problems with them prior to the meeting, we have seen in the weekend, prior to it that they were rebooting at least once a day but that was during the night. However, on Monday, you might have noticed a hiccup around 17:30, when they spontaneously rebooted and since then, we just did it every morning just to be sure. We haven't got to the root cause of it. We didn't have the time for it. But we're going to investigate it as soon as we get back to our normal life in the office.
On the wireless network. We have been using Ubiquitis as well. We deployed 55 of them across the area. You can see on the right‑hand side, that the vast majority of them are deployed here in the main room and the side room. But we also cover the whole way and all the meeting rooms, the little meeting rooms that have been used, including the tutorial room.
In terms of connected devices, we have seen a bit over 800 simultaneously connected devices at any single moment in time. So this is not a total of unique devices we have seen across the week but just snapshots in time. It's very nice to see that the vast majority of you connected over 5 gigahertz network. And the number of 2.4 connections are just dropping steadily.
I'd like to zoom a little bit on the NAT 64 network that has been used as well, around a peak of 100 users. And the question we were wondering is, we get too little feedback from you on this network, if there is anything that you still see that is not working or is not working properly on it, please let us know. So, your feedback is essential in trying to shape out our next configuration. So, please send us feedback on what works and what doesn't.
This is, let's say, a wireless roaming structure, so you can see the foyer area, which is the lower band, and that one has continuous population traffic on it, but you can see it growing in the breaks and especially during the lunch time. And the pattern is absolutely as expected. This is basically a zoom on, let's say, Wednesday, one entire day from 8 in the morning until 18 in the afternoon.
RIPE network app. You have been using that, thank you very much. This is the number of participants from RIPE 75 to 6 and 7, so it's steadily growing, and the number of messages being sent and ‑‑ well, meetings not growing, but the number of messages grew quite a bit. So, looking forward for the next meeting for it to be used even more. Please send again feedback on your experience with it to our team.
We have been using, we have been following discussions in the DoT and our resolvers actually listen to port 853 and we have been running the Knot‑resolver since last meeting when the Android Pie was still under development and now it's been released and actually we saw some clients connecting to it, trying first the 853 port which is something very good. We'll continue to use it.
On the team:
You might have not noticed, but the wonderful stenographers have been with us for a decade now.
They mentioned that in ‑‑ normally, they use a dictionary of shorthand strokes of about 65,000 common English words. In addition to that, they have built extra 36,000 specific words with technical acronyms and technical terms and names. So this probably makes them the most experienced technical stenographers in the world, at least in our area.
The web services team is being great, they do everything not only the RIPE networking app, but also the synchronisation of your presentations to the presentation machines, they upload presentations and plus the entire website that serves this meeting.
And finally, the operations team, it's not just us, we're the sixth people that built this in the weekend prior to the meeting. But the network, also the people in the office was Brian, Kaveh. Of course, Martina was just our guardian angel here taking care that we have all the conditions to do our job, so thank you very much to all of you for coming here. Please continue to use the network and send us the feedback with any kind of improvements, suggestions and comments you might have. Thank you so much.
PAUL WILSON: Paul Wilson from APNIC. Congratulations on the technical infrastructure, it's been fantastic. Twitter is geolocating me on this network in Marseilles. So that might be something to look at. Thanks.
BENNO OVEREINDER: Now, it's time for the last three presentations, the lightning talks. First, Amanda Gowland will give us an update ‑‑ well, update, a short lightning talk on the Women in Tech Lunch and what we have discussed.
AMANDA GOWLAND: So I have been to 17 RIPE meetings and this is my first time on stage and I think that this might be the most relevant least technical RIPE meeting presentation ever.
And there are no acronyms at all in this presentation.
AMANDA GOWLAND: You may notice that RIPE meetings are growing. 20% newcomers, which means that one in three people are here for the first time. We have, I think the latest figure was 801 people checked in. And yet there are more Alexanders, Christians, Daniels, Johns, Marcos, Martins, Pauls, Peters and Robs and Thomases than there are women here, and that's based on the median of the gender metrics that the taskforce is gathering. It's not scientific, but I calculated around 13%.
Overheard in the foyer on Tuesday: "Are you coming to the Women in Tech Lunch today?"
"No, why would I ever want to go to that?"
What the Women in Tech Lunch isn't, is this, or this, or this.
What it is, is an open and inclusive event where everyone is welcome. A huge success. It's the third time we have done it and every time we have done it the rooms have gotten bigger and every seat is filled. It's a place to have productive, honest discussions about how we can make our meetings more inclusive.
And at RIPE 77, we had an expert panel and group discussion on two topics, quotas harmful or helpful. There wasn't really consensus. There was range of opinions among both genders in the room and male allies, how we need them and how to find them. And this event was sponsored by NetFlix, we we really appreciate. Here is the panel. So these were people really outside of the community who were diversity and inclusion experts, and we were about 40% men, 60% women. So it was a good crowd.
I want to take about male allies, because that's something that everyone in this room can be. It's a member of an advantaged room committed to building relationships with women, expressing as little sexism in their own behaviour as possible, understanding the privilege conferred which their gender and demonstrating active efforts to address gender inequities at work and in society.
There is a range of what it means to be an ally. So ranging from indifferent, so comments like why would I do to that? So understanding that we're talking about these issues, to participating when you are asked, and then to advocate, and that's what we want. We want advocates to routinely proactively champion gender inclusion, and it's something that everyone here can do to make these meetings and their work places more inclusive.
And a really good easy way to start this is just to talk to the women in your circles, in your work places, in the meeting, to talk about their experiences, because they are different from your own.
And, of course, come to the Women in Tech Lunch at RIPE 78.
So that hopefully, next time, this is what we hear:
"Hey, are you coming to the Women in Tech Lunch today?"
"Of course, I am bringing a few colleagues with me too."
There is some other things that we're doing, it's not just about gender but embracing different kinds of voices in the community. We do the RIPE Academic Cooperation Initiative, so we bridge the gap with the research and academics. We provide travel, accommodation and the meeting ticket and they present their work to the community. The other thing that we do to remove the financial barrier is the RIPE fellowship. This is open to anyone in the RIPE NCC service region and we also fund your trip to the meeting.
And we are seeing new faces in this meeting, and it's really nice to see.
The other thing that we have launched, it's sort of a soft launch in August, is the RIPE meeting mentoring programme, it's a really easy way for people to pay it forward and help newcomers to have a more enjoyable experience here. We match mentors about 2 or 3 weeks before the meeting, based on interest and expertise, and it's resource‑light, so it's an e‑mail with someone before the meeting to figure out what they want to get out of it, having a coffee with them on Monday and maybe checking in with them throughout the week. And before this meeting we had about 13 people from the community, put their name on the list, I'm putting this link up here and I would expect that in a couple of days time I will see a lot more names on there hopefully and if you are also in need of a mentor you can go there and put your name on the list as well.
So, saying ah to diversity, for me, is like saying oh to really cute animals, and Bill Murray, because there is nothing bad about Bill Murray.
When I was making this talk I was called brave for doing this. And the fact that talking about diversity in this meeting is an act of behave re means that we have work to do. Because there are incredible women in this community. But, many don't come to these meetings and many don't go to the mics or submit presentations. And talking about how we can have more diversity in this room, on the stage, in the PC and in the Working Group Chair collective, on the mailing lists and on the mics will only make this community stronger for everyone.
Thank you.
(Applause)
BRIAN NISBET: Thank you very much. And now I will crudely take the microphone back. We have a couple of minutes for questions, and only a couple of minutes, so...
AUDIENCE SPEAKER: Hello. My name is Alexander, and for the purpose of this meeting and the registration system, my gender is not specified. I must confess that I have not to use my gender during this meeting in any form, but what I want to say about this presentation: First of all, in slides 18 to 24, you say that women don't come to ‑‑ don't submit presentations, something like that.
AMANDA GOWLAND: I say many.
AUDIENCE SPEAKER: But I know much more people who don't do this because they don't speak English. If someone is not going to the mic, is not submitting presentation, is not participating in mailing list, it's not because of gender.
AMANDA GOWLAND: In your opinion.
BRIAN NISBET: There is people at the mic ‑‑
AMANDA GOWLAND: There is people at the mic I have never seen before.
AUDIENCE SPEAKER: Sara Dickenson from Sinodun. Thank you very much for doing the talk and thank you for the RIPE efforts in this direction. One of the reasons we identified at the women in tech meeting as to why male colleagues were confused about its purpose was its name, that it was called Women in Tech rather than diversity in tech or something similar. But I was told that was a very conscious decision and I'm wondering if you can speak very quickly as to why it retains that name.
AMANDA GOWLAND: So, when we were first talking about ‑‑ and I am going to talk about the diversity taskforce although I'm not presenting on behalf of them. Gender is something that is an easier thing to tackle and there is intersectionality, so when you make the meetings more inclusive for women, you also have a more inclusive meeting it kind of opens it up to everyone. So perhaps at some point, the title will change, but I think it was an easy way to say this is a place where women and men can come to talk about the work women are doing.
BRIAN NISBET: Just to be clear, there are two people at the back mic and one person at the front. Unfortunately, that's it.
AUDIENCE SPEAKER: Salam Yamout. First, I'd like to thank you very much from the bottom of my heart, you, Amanda, and the other members of the Diversity Working Group, for all the efforts you are making. The tech lunch was very interesting and very useful for those who attended. I can speak from a double minority point of view as, being a woman and a Middle Eastern, yes, we have a lot of work to do and we should not be shy anymore of talking, naming things what they are they are. Yes, there is a problem including women. Yes, there is a problem including Middle Eastern and other developed countries representatives, etc. So let's call things because, together as a community, we can do great things. We have solved a lot of problems bigger than this one, so if we get to it, I think we can do something together.
AMANDA GOWLAND: Thank you Salam. Can I just quickly in response to Salam. I think, you know, people are talking about the work that we're doing here and other circles and other meetings, so I think we have a really unique opportunity as the oldest RIR community to really set the bar high so that people say, hey, look what RIPE's doing, isn't that amazing. It's really good for this community.
AUDIENCE SPEAKER: Sacha Pollard, just speaking for myself. I wanted to thank you very much, as a father of a son on the two girls that are technically interested being 10 and 13 years old. I absolutely loved it and whatever I can do to support, I'd love to do that.
BRIAN NISBET: Join the diversity task force mailing list, talk to us, help us out with that.
AUDIENCE SPEAKER: Anna Wilson. I think you and I both know that brave is a double‑edged compliment. To those who make that compliment, thank you, try being brave.
(Applause)
BRIAN NISBET: And so to move on to a lightning talk with lots of acronyms in it.
SANDER STEFFANN: A little bit of background. There was a draft in the IETF. It's in last call now. I had strong opinions on it, but of course IETF said you are just one person, you can't speak for anybody else. So therefore, I am presenting it in front of this audience, and I would really like your opinions. I think the IETF needs them. So I'm trying to keep this as neutral as possible, I'm not going to state my personal preference, but please do state yours on the microphone.
So, this proposal is a flag in route advertisements that say this is a v6 only network. There is no v4 here. So for us IPv6 advocates that sounds pretty good. But what's the actual effect of this flag?
It means to say, to tell hosts please turn off your IPv4. Don't try to do RBGP, all that kind of stuff. There is only v6 on this network, kill your IPv4 stack.
And the reason for this is that some people got annoyed by all the Layer2 broadcasts and packets. ARP is very noisy, people constantly requesting DHCPv4 when there is nothing on the link, Multicast DNS is specifically mentioned as a serious reason for this. Also, all the wireless controllers need to keep track of who has which v4 address for the optimisation, so they would need less state. And of course, lots of devices are battery‑powered, so them constantly trying to get the DHCP address or getting that broadcast from ARP drains their battey. There was even an argument that said v4 might be used maliciously on a v6‑only network.
Now, there is a scope to this, so it's ‑‑ this is to limit its impact. Only a router advertising itself as a default gateway can set it. And all different gateways must set it. If one gateway has the flag off, it should be treated as off. This is, for example, intended for if you already have a v6 network, somebody else can't just send a rogue RA and turn v4 off, because the other v4 gateway will say leave it on.
It's only a recommendation to a host if a host really wants to they can leave v4 turned on if they want to, but the intention is to kill IPv4 on the network.
So, some other options have been proposed. People say like in enterprises you can just turn off the v4 ethertypes, so why do you need a flag to kill v4? Just block it on your switches. There is a DHCP that says employees don't use that, that's their name the 169.254 whatever addresses for our configuration. People in enterprise might have a central management in their Windows policy to say turn off v4, so why don't we use those in why do we need a new flag for this?
But the discussions, it turned out people said no, no, this is not sufficient, we need something extra for this.
So, why wasn't it sufficient? Well, you know, the operator might not control the switches, they mightn't be able to block the ethertype or might not have central management for the hosts and for the DHCPv4 option, well if you want a v6‑only network, we don't want to have implement a server because why pollute our code with v4 stack if we only want v6.
On the other hand, there are arguments against this. Who actually has a network that has no control over the switches and the hosts, doesn't have a DHCPv4 server and has a strong desire to actually disable IPv4. For example, most of these things might be valid for a home network, where somebody doesn't really control anything, but does a home network really care about some background IPv4 traffic, so there was a question already about is this even necessary? Actually a use case? Also the benefit and risk factor. Like, from my point of view, everything that is added to a protocol people need to learn about. This takes up brain capacity and my brain is getting quite small, so it must be the age.
Principal of least astonishment. One protocol killing off another protocol might cause some surprises. There are some security risks, what have you v4 only network that doesn't have a v6 router. That would be the only default gateway and it can switch v4 off. At that point you could, with one device on a home network, you could basically bring down the whole network.
And people say also, what does a flag saying something about v4 do in IPv6 anyway? It's in the wrong place.
So, the status is it's in 6man. It's a proposed standard and it's currently in Working Group last call. And I would like your opinions on this.
BENNO OVEREINDER: This is a lightning talk. So please ‑‑ we all want to have some spare time about an announced thingy with people ‑‑
AUDIENCE SPEAKER: So in the true tradition of these things ‑‑ I am Remco ‑‑ I'd like to pose a comment as a question. I think that, in addition to this standard, we also need an expansion of the DHCP for IPv4 that says don't listen to weird stuff that IPv6 ‑‑
SANDER STEFFANN: Can you add one for DECnet and ‑‑
RANDY BUSH: Remco got my first half. And the second half would be, could this please work remotely because it's a wonderful attack investigator and I'd like to be able to do it over the net.
AUDIENCE SPEAKER: Ignas Bagdonas. What is the status of running code and deploying experience of this?
SANDER STEFFANN: I don't think there is any.
IGNAS BAGDONAS: Don't you think it's a bit premature to put in a last call?
SANDER STEFFANN: I talked to the Working Group Chair of this Working Group and it is currently in last call and it's not my decision. Like I said, I am only the messenger here and I thought people should be aware and IETF should get feedback from the operator community.
AUDIENCE SPEAKER: As you might know, we are defender of the default gateway and this time we are default stack and we want to keep it that way so absolutely no intention to go to this suggestion because we are afraid you are breaking a lot in consumer market because this is all enterprise orientated.
AUDIENCE SPEAKER: Peter Hessler: As a host operating system, we don't see the point. We think this is a silly thing and this should not be a suggestion that is just randomly declared by some random device who is sending who knows what's going on. Like block this at the switches, install RA guards, install DHCP guards etc.
RANDY BUSH: Is this any operator who will speak for this?
AUDIENCE SPEAKER: Tasha, German government. I just want to propose a name for it. It's a denial of IPv4 flag.
SANDER STEFFANN: To be fair, when I read this, I thought the whole idea was insane. But, I promised the Working Group Chairs of 6man that I would stay neutral in my presentation to get honest feedback from the audience, but I can see where this is going.
BENNO OVEREINDER: Okay. Thank you.
(Applause)
Next speaker is Toma Gavrichenkov on wrong, wrong, wrong methods of DDoS mitigation.
TOMA GAVRICHENKOV: So this is a quote from one of my favourite bands. For those of you who don't know who it is, it's it's Dave Gahan from Depeche Mode and he is the living proof that you say the word wrong 65 times in five minutes and still be a rock star.
Let's see how it works for me.
One thing I know about people is that they just love geography. All those fancy maps and plans and this is probably the reason why, when I divide through the security part of stack exchange, I frequently noticed one common suggestion about preventing cyber attacks.
So, say we, as a company, don't expect Spanish and ‑‑ Chinese customers and why don't we just deny access to the IP addresses, right, we are not Facebook, after all, your services are available in limited regions only, and even Facebook is not available in China, then why to care?
Well, outside the IT world, such technique is sometimes referred to as red‑lining. It's basically like you literally draw a line over the map. And they say that you don't offer your services outside the resulting shape, because you believe it's not safe or cost effective for you to do so. And we can do the same for IP addresses, right.
An IP address belongs to some country and there is an official database supported by IETF and RIPE which provides some mapping between IP addresses and countries. Wrong!
If you even stick to the technical aspects for now. In fact, geodatabases are commercial products produced by commercial entities and gathering data in obscure ways, they might be use for statistical or research purposes but providing a service using those is plain stupid due to countless reasons, in the IPv4 world due to the address space exhaustion, IP addresses get leased, bought and sold very frequently they migrate across regions and they sure surprise are not even bound to a single region and may be announced from numerous regions at the same time in which case a geodatabase would just give you some random response.
And fortunately, RIPE and databases are not significantly more liable also due to the same reasons.
As a side note, your copyright law enforcement just got way more complicated.
Well, some of the above, but not all, might be better with IPv6, but as we speak directly of DDoS attacks to IPv6 DDoS error is something yet to come and as we collect experience there might be different issues around so it might be a bit early to speak about it.
But speaking of DDoS, what if we know for sure that the remote party is doing malicious stuff? So here is is our set‑up, 40 gigs of traffic, apparently DNS traffic since it comes from port 53. As we know, such a pattern is typical for DNS amplification attacks and amplification attacks uses vulnerable servers on the Internet to, you know, amplify traffic. So, the source IP address in those packets are the IP addresses of vulnerable servers.
What if we use whatever technology we have to deny access to those vulnerable servers only? After all, let them finally patch all those DNS servers so that they won't be a direct threat to us, right?
Wrong.
This is a true story, the events depicted took place in 2013. At the request of the survivors, the names have been changed. Out of respect for the dead, the rest has been told exactly as it occurred.
A company got those gigs of DNS traffic and decided to handle it with block lists. After two hours, the attacker noticed that somehow and immediately changed the attack pattern. The ability to run amplification attacks before was based on the ability to generate packets with spoofed IP sources. So, that was what they did again, but in a different way.
They have started to flood the victim directly with source UDP port 53 and fake source IP addresses from the whole IPv4 address pool. The network script company decided it's still an attack and banned the sources.
As you may enumerate the whole IPv4 address space in hours, as you may guess, it didn't take too long for the network equipment to run out of memory and shut down completely.
To adds to the injury, the attacker has started with some of the end user access prefixes of ISPs most popular in the country. So, the access to the website was suffering long before the complete down time.
So, here is our lesson: don't populate block lists automatically. If you haven't verified the source IP address, especially when it comes to reflection attacks, they might not be what they seem to be. And then one question arises:
What if we say, well if I at least ‑‑ that there is actually an amplifier on the remote side? Let's scan the Internet, collect the IP addresses of potential amplifiers and if we see them in the packet sources, let's block them because in any case they are amplifiers, right? Guess what, wrong.
There is again numerous reasons why you shouldn't do that, again given million of potential amplifiers on the Internet and again you'd be easily tricked into blocking excessive amount of addresses but the thing here is there is not only potential for false positive but also of a false negative. Here, due to guess what? Network red‑lining on the other side. People usually hate scanners. They block them because they are shiny equipment marks those as a direct threat. They block them in numerous ways, poor scanners and attacker could use some of the amplifiers you just won't have access to. Nobody is going to ask you before if your scanner is a nice and quiet. Ideas doesn't have a concept of nice scan.
So, here are key points:
Don't do block listing before you are sure that the remote site is not fake.
Do not apply block listing where it's not necessary or prototype to do so and finally stop please breaking the Internet in ways it wasn't supposed to break.
(Applause)
Remember, a complicated solution is usually better than a simple one because simple solutions have complicated consequences. Thank you very much.
(Applause)
BRIAN NISBET: I think Sander's flag might be good for some of that.
SANDER STEFFANN: It's not my flag!
BRIAN NISBET: We have a couple of minutes. So please...
AUDIENCE SPEAKER: Jim. Just more examples because now all this blacklisted red‑lining, no matter how it's named, it's expanded more on TLD services, registrar services and so on. It's very wide. And it's ‑‑ I can ‑‑ it has no answer. Because you imagine that white‑listing it will create the list of goals for attacks. How to drop down Internet. Where for me it's interesting ‑‑ your advice was be careful, but unfortunately a lot of people use different operators, and so on, as a source.
BRIAN NISBET: We have a minute.
AUDIENCE SPEAKER: Just might not all people know that I'm identified by MaxMind, and I know what Iranian feel after dissensions. Thank you.
TOMA GAVRICHENKOV: If this is a question, then yes.
PETER HESSLER: Do you have an opinion on expiry times for a block list? IP address gets added, should it expire? Do you have a rough recommendation expiry? As you mentioned earlier, it may be transferred earlier.
TOMA GAVRICHENKOV: So ‑‑ and you are following the advice of identifying the source IP address.
PETER HESSLER: Yes.
TOMA GAVRICHENKOV: I believe some ‑‑ like, we use it up to no more than 8 hours, but usually much less.
AUDIENCE SPEAKER: That's actually a comment. There is a very common quote that everybody knows and they always attributing it wrongly. It says for every complex problem, there is a solution that is simple, neat and wrong. The quote is, explanations exists and existed for all the time and there is also a well known solution to every problem which is plausible and wrong. Just another proof of that and thank you for your talk.
AUDIENCE SPEAKER: Daniel. Internet citizen. This was by far the best lightning talk of this RIPE meeting. Very good style. So here is my question: Will you come back and talk about something else with the same style at the next meeting?
TOMA GAVRICHENKOV: Just to make it 100% sure, are you going to pay for this? Let's talk about it later.
BRIAN NISBET: Okay. Thank you very much.
So... at which point in time the PC will relinquish our control of the Plenary and hand it back to the RIPE Chair. So, Hans Petter, please, and thank you.
HANS PETTER HOLEN: So. Thank you very much. It's been an exciting week with the really, really exciting programme. I realise, after going to RIPE meetings, how much I don't know and how much I still have to learn, and that's one of the really good thing with coming to these meetings.
So, checked in: 796. And then I think somebody sent out somebody to get another 5 registrations so I think it's now 801 registered for this meeting. That's a new record. And 226 newcomers, so a third newcomers, this is really great.
So next venue, we will have to look for...
I don't make these slides, this just appears from somewhere.
So, you can see the increase here. It would be exciting to see this number on the next meeting and the projection through times. We had a BoF on this on Tuesday night where we discussed a bit about what we want the community to be in the future and what we want the meetings to be like. And I think that's a conversation that we need to continue.
6% more newcomers than normal, which is really good. But I think that, as a community, it's also important to think that we need to retain people. We need to come back, because otherwise it won't be a community. So, we need to have the right balance here and I think we are doing really good with having new blood in but also a large base of returning people.
The split is not quite as usual, it seems that we are now attracting more government people and I think that's good. We need to realise that we are actually living in the real world, not just in cyber space, and figure out how to interact also with the governments so that we can make sure that we still have an open Internet and that governments don't do things that is not the best way to solve problems. And we need to help the governments to solve the problems on the Internet so that we still have a free and open Internet.
So, I'm glad to see that we now have both the Netherlands and Germany as the two biggest countries in Europe and the US has been reduced to the third biggest country in the RIPE region. Sorry about this, ARIN, I guess we're taking away some of your attention. But I think this is a great message to us and a great message to the success of the community and actually to the PC, that we do attract people from outside the region and they don't come because they love Amsterdam, they come because the content the PC puts together. At least that's what they are telling me.
(Applause)
So, the meeting wouldn't have been the same without the Working Group Chairs and the Working Group Chairs is also pulling some log between the meetings because there are mailing lists where the discussions are continuing after we met here. So here is a picture of the Working Group Chairs. There was nobody leaving the group this time. There were two who were up for selection, but the Working Groups picked them again so there is stability but one of the Working Group anti‑abuse figured out that three is better than two, so al I remember resume a was appointed as the third Working Group Chair for anti‑abuse. So a big hand.
(Applause)
And of course, as always, thank you very much to the RIPE Programme Committee and I think do we have some gifts for the Programme Committee. So if you could come up here, please...
Thank you very much everybody for putting together a great programme.
(Applause)
There is also been an election of two new members of the Programme Committee as was announced earlier today, but here you have the names and pictures for the record. So welcome to Maria and Alireza to the Programme Committee.
NRO NC, so as you all know, we have the Number Resource Organisation Number Council, the NRO NC, and the NRO is acting as the supporting organisation when they go to RIPE meetings, plug in the socket as Jim said this morning, so the NRO and the ASO is the same thing so the NRO NC is the same thing as the ASO AC. If you are confused now, that's the purpose.
Now, what there isn't confusion about is that Nurani was re‑elected to serve on the committee for another three‑year term. So a big hand for Nurani.
(Applause)
There has already been a lightning talk about diversity, about women in this community, and this is something that's close to my heart as many of you know, I have two daughters and they are both entered into this industry, one on the lower layers doing fibre connections and so on, and another one as a programmer, and this is not my choice, this is their choice, and I really think that we, as a community, need to make sure that we attract all talents from all parties of the population. If we leave out women, we leave out 50% of the population of the world. That's not a good thing for attracting the best talent. So I'm really pleased to see that the diversity task force is working ahead. Gathering metrics, there has been on‑site childcare, and we need ‑‑ we have done some reach‑out to local communities.
The next action that we have put on their list is to tackle gender diversity in speakers, there is still some way to go there. There is also thoughts on updating the code of conduct. And there was a proposal that will, in discussions with the PC, maybe test it out at the next meeting, rather than using the microphones for questions, using an app for questions to see whether that will attract more questions from the general audience.
So that's the short report from the diversity task force.
We also had an accountability task force, and there was a presentation on Tuesday by William, where he presented the draft report from the task force. And they are still soliciting comments to this report, so please send your comments to the list by mid‑November and they aim to produce the final report by mid‑December.
And this is really a great undertaking done by a task force in the community, through bottom‑up spirit, so that we've described our accountability mechanisms and framework and they have identified some areas for improvement, but that will be for the community to address after we have the final report, so I'm looking forward to that.
Chair selection process. Some people are asking me, are you leaving? Well, maybe some of you want me to leave. So that's why we're putting in place a process so it's well‑defined how to select the Chair.
There was a draft put together that was posted to the list and some of us were really naive and we thought that this was based on the discussions at the last meeting, so we will easily have consensus. Not so. And that's really good, because that means that you actually read these drafts and care about them. We had a discussion this morning. There seems to be consensus now on the role description or the function description, to be precise, of the RIPE Chair. So what's been done here mainly by Daniel, but also by great input from the mailing list, to document what the Chair is today, based on the history and what the Chair is today. That doesn't prevent the community on what you want the Chair to be in the future, but at least we have a document that describes the present.
So, that is sort of document where I assume that we now have something close to consensus, or I want to call for consensus here and on the list for that function description. But for the Chair selection process, there is a clear need for more discussion. There was discussion this morning, and there is clearly a need for an updated draft. That will be done by Mirijam and those who took notes in this morning's Plenary and sent to the group. And I'm not sure there will be consensus on that draft either. Therefore, I ask you to read that and comment on it so that we can refine it towards something that we can achieve consensus on. This needs to be a community effort. And there is a separate mailing list set up for that, so if you are interested in this, please join that list.
Here is a brief overview of what the RIPE Chair is doing today. And I will post a message to the RIPE list just after this meeting, calling for final consensus on the RIPE Chair function description.
Any objections to that?
So what did you think of RIPE 77?
(Applause)
In addition to that, you can go to this URL and give feedback.
So prizes. Rather than picking the first registration that we have done for a long time, and for some reason it seems to be the same person or persons every time, there is now a new algorithm. It may have been developed by a Secret Working Group, it appeared by magic. So do we have registration number 77? Why 77, is there anything special with 77? It's on your T‑shirts, that's why. Yulia Makhlin. So will we accept the proxy? Okay.
And then, not the second prize, but maybe the first prize is for the first ‑‑ oh, no, that was for the first newcomer. So this is how to trick me, right, you give me something on the slide and the opposite order on the paper. This is brilliant.
Since several meetings ‑‑ I think maybe next time I'll ask for the scribes to be displayed in front of me so I can read what I'm supposed to say.
HANS PETTER HOLEN: So, first newcomer ‑‑ number 77. So can you please look on your badges to see if you are number 77. Paul Coerkamp. He is not here? So, 777? Andrei Robachevsky. He is gone? I'm not sure this new algorithm works. Number 7 then, Juri Bogdanov.
I wonder what the next slide is. And that's thank you to your sponsors. These meetings would not be possible without the sponsors. And by that, I will welcome you to Reykjavik, Iceland, in May 2019, and by that I think that's it.
HANS PETTER HOLEN: Thank you very much everybody, and with that you can now go for lunch and safe travels home and see you all in Reykjavik.
LIVE CAPTIONING BY
MARY McKEON, RMR, CRR, CBC
DUBLIN, IRELAND.
