IoT
18 October 2018
At 4 p.m.:
JIM REID: Good afternoon everybody, ladies and gentlemen, boys and girls, welcome to the second meeting of RIPE's IoT Working Group. This is actually quite interesting for me, because I think it's interesting that we have got a new Working Group, a fairly new Working Group and we are actually got new presenters. So we are getting lots of new people coming to the community for the first time, to present, from a combination of different backgrounds. I am delighted we have got representatives from the Dutch police here, to give their perspectives on the concerns around IoT.
I think it's very good to have this kind of dialogue between the technical community and law enforcement regulators and that is one of the things I think this Working Group should focus time and energy upon. We are going to have a remote presentation from Michael Richardson, from Sandelman, from Canada. We have never tried that before and so I hope the network Gods work in our favour and all the video and audio casting and everything will work out perfectly. I'm sure the RIPE NCC's meeting team are doing a fantastic job to make that happen, right now.
So before we get started, just one or two reminders. Please switch off your mobile phones, or put them onto silent mode, if you possibly could. If you are going to come to the microphone to speak, state your name and ideally, your affiliation. This is very important because remember the session is being webcast and the people who are following remotely, either through the webcast or the Jabber room, would like to know who it is that is saying whatever is being said at the microphones. So with that, I am now going to hand things over to Marco Hogewoning, who's going to give us an update on what the NCC's been doing recently, in the wild and wonderful world of IoT.
MARCO HOGEWONING: Thank you, Jim. Good afternoon, everybody. My name is Marco Hogewoning, as Jim said, I work for the RIPE NCC. I am the, lets call it, the topic lead on IoT, I am tasked to try and make sense of what goes on here, follow different groups, I am involved in ITU, Study Group 20. We're a member of the Alliance for Internet of Things innovation, in Europe, so we kind of try to track what happens in the different fora, in the different groups.... what is being said. And then collect that information and bring it back to the community, where we think it's relevant. So for instance, also what we did more recently was; There was a RIOT conference, the people from RIOT spoke in Marseilles. You might remember them. They had a conference in Amsterdam. We attended that. We sponsored that. We helped that. They are not here, right now, but they, earlier this week, published a report of that conference and I will also share some of my observations there.
So, wandering around IoT world and looking what goes on, what is the key topic. You guessed it, in the ‑‑ security, yeah, no surprising here. It's actually always all about security, whether it's privacy, or whether it's disruption of services or infrastructure, the key word really is, here, security. And while I do notice a bit is we, from our technical background and from our community generally consider it a layered model. We often say that is application space. That is not for us, that is the transport layer. We just get the packets from A to B. But in IoT world, they sometimes skip a layer and there is much more integration. It seems to be a much flatter pancake. As you will see this coming presentations about, for instance, MUD and FUD, people are really looking at what the transport layer can do in securing the applications and securing the data and that's I think something to keep in mind.
The other big trend and it has kind of started seeing come through, in maybe the last half year, last year, is that we see more and more talk about safety, and it seems like a pedantic linguistic exercise, but people, more and more, become aware there is something about it. There are lives at stake, sometimes, when you think about self‑driving cars, but also in other parts of the infrastructure. Safety becomes more important. And where it also really becomes important, if you talk to the people that are looking at industry, for[ ***], the people look at manufacturing etc., where health and safety is a common thing. You have to make sure that that machine doesn't kill your workers, or you. And in that context, cyber gets a much more prominent role. What if my factory is online, what if some punk remotely gets hold of my robot and the robot kills one of my workers. So there is quite a thought there, in like: 'Okay, what does this imply for our liability?' What does this imply for us? The other big thing that you hear and I also heard that during the RIOT conference a bit, is that the bigger industrial players, basically, all come back and say 'We are not afraid of regulation, we want that clarity, give us the guidelines, create the level playing field, that helps us, the lawyers, the insurers, that helps everybody to figure out what goes on.'
So, we end up in health, safety and security. I think that's where it's going. And that's important, because it's also what we have been advertising. Don't create this IoT silo. You have to get that security awareness, integrate it in your main processes. It has to be everywhere. You can't just create this super regulatory body. You can't do everything. You have to do it in the sector itself.
So that's the main trend. Now, to give you an example on how this kind of could work and this is one I came across a few weeks back, at a conference. I immediately was like, yeah. We spend sometime with them, they start to understand what we do, let's call it hacking regulation, hacking in the sense that retooling old to do something that it was never planned to do. And a nice example here is RED, or the Radio Equipment Directive. You can quickly google it, it's 2014/53/EU and mandatory requirements for everything that has a radio. The basis in there, is indeed, health and safety. Make sure that whenever you use a device it doesn't fry your brains out. There is also, so of general understanding, interoperability, robustness, all that kind of stuff. Importantly, it will apply to everything that has an antenna, whether you actually use it, or not. A nice example here is, for instance, a printer, with USB, ethernet, WI‑FI and Bluetooth, because it has a WI‑FI chip, RED applies. The compliance check is done on a self‑assessment. It comes with the C mark, so you have to kind of check yourself, whether you're compliant. Of course, exposed, if there are compliance, or people don't trust you and maybe sometimes consumer protection groups do that. Third party testing in labs could mean that you are not, you are not compliant. You have to fix that. The ultimate penalty there is that you are forced to do an EU‑wide recall of all your products, put the advertisement out, if you have one of these bring it back to the shop and get your money back. That is a massive penalty. That can get costly.
Interesting bit, RED and the Internet of things, if you go to the text of the directive, it's not too long. You see that it has a few optional requirements. There are more. I have listed a few of the more important ones, I think and these things are not filled in yet; they can later be activated for specific product groups. So, for instance, do not harm network, or misuse network resources. Anybody reads DDoS there? Protection of personal data and privacy. Of course, GDPR, here we go again. Only compliant software that can be loaded. That needs a bit of explanation and understanding, but kind of tries to say is that, as a manufacturer, if somebody meddles with your software, you have to make sure that your device stays within the compliance limits. All of this can be activated and it's processed, but it's important by what I call delegated act. The Commission can kind of quickly decide to activate one of these paragraphs and further develop it for a product group, without lengthy consultations with parliament and Member States. This is: 'When I snap my fingers, it's there!' So here's the thought, and this is real chatter, we haven't seen text yet. That's a disclaimer. Activate some of these paragraphs, specific for IoT and the first round, that is what everybody is talking about, will likely include smart toys and smart watches. The smart watch was a big of head scratcher and I explained what goes on, it's a case in Germany. Basic thought is activate these things and, for instance, enforce regular software updates. Either every once in a while and when known vulnerabilities exist, you have to fix your software, but they are also thinking about, for what lifetime? Two years, four years, ten years, what is expected lifetime of a connected toy? The discussion becomes who is responsible, the manufacturer, the distributor, or the retailer? And the RED directive takes quite a broad approach in looking at the whole value chain and kind of says, like, you go sort it out. The retailer is partially responsible for making sure that the C mark is there, the distributor, etc..
So, I thought it's interesting to see how they now apply something that has been around for ages, radio compliance and bring cyber security and software maintenance into the dialogue and it kind of ‑‑ it's kind of makes you think: What is the impact for the RIPE community? Because probably a lot of you are distributing things that have radios in them, you know, average cable or DSL modem, the setup box and one of the things I came across ‑ It says like: 'If you provide your custom software package, if you, for instance, give your customers an OpenWrt packet and flash modems for them, the directive probably assumes that you are the manufacturer. It means that you are responsible for compliance and any text inserts, they say like, you should do this for the next ten years, means you will be maintaining OpenWrt for the next ten years. As I said, we haven't seen text yet, but watch this space and people are looking in similar directions, for general product safety.
That's all I had on my list, in sort of the observations. I am happy to take questions now. Otherwise, find me, or mail me. My address is on the screen. Thank you.
JIM REID: Thank you, Marco.
(Applause)
Are there any questions? Well, I have one if nobody else has one. And you mentioned that the directive has got some empty space to be filled in later. How could we participate either in commenting on what could go into that empty section of the document or involved in the consultations about whatever comes out of the Commission if and when they finally finish the text, what is your take on that?
MARCO HOGEWONING: The short answer is I don't know because I don't know how delegated actually would work in terms of consultation. I know that the Dutch radio regulator is looking in it and they are talking to their German colleagues and I think France is involved so very much talk to your local regulatory agency and sit with them and explain the complications, I think that is the shortest thing. The Commission can act pretty quickly if they want to.
JIM REID: We have some people at the mics.
PETER KOCH: Thanks for that update and I think it's important to highlight the Red actually. While you brought it up as an example or blueprint in a way, it is also if you go back to the reports approximate the Red that there were certain drawbacks and large concerns regarding open source projects and you mention OpenWrt already and the requirement that every software version needs to be certified is a bit at odds with regular updates especially for security purposes. Could you add something in that direction, what is the awareness there? What are the ideas with, say, exemptions or special treatment for open source projects?
MARCO HOGEWONING: We were in the same discussion, indeed it was raised so I later on read the text and disclaimer, I am not a lawyer so please consult your own legal department about this. The way I interpret it is that they are fine with open source software, there is no limitation there but as a manufacturer you should safeguard that even if somebody gets to your software the device stays within limits, for instance that the chip at some point says sorry, I am going to ignore the software, you can't just pump up the volume. Also if you are really an amateur hacker, amateur radio enthusiast you are exempt. There is a specific exemption for amateur radio, but that probably doesn't cover the fact that you are shipping OpenWrt images to your customers.
PETER KOCH: Somebody's exception is another one's fine so that is of course itchy but thank you.
BRIAN NISBET: HEAnet. I may have missed this and apologies if I did, but some of those pieces which can be activated by the delegated act seem to me like things that should be there right now for various things and I am wondering do you have any sense of why they are waiting for later in regards to things like network abuse and things like that?
MARCO HOGEWONING: Yeah, it's very generic but I think this originally came from when they say network abuses stick with inference since your regulatory limits on spectrum you can only send this amount of power out so that general sense of provision is there but it leaves it open to further specify specific cases for specific product groups so there is a bit of content there but it's very superficial, very broad and I think in the sense it's like yeah, data privacy is covered by GDPR now so it's not overreaching but they have the capability of putting specifics in there.
BRIAN NISBET: That makes more sense, thanks.
Christian: From a machinery company can. Is there any discussion if everybody needs to be connected? In my opinion, let's say in the past there were a lot of software packages don't use it for whatever life influencing, mean this software is not allowed to be medicine equipment or things like that. Now we try to connect things which get unpatchable forever. So, I think there should be also a discussion that not all that is possible should be allowed.
MARCO HOGEWONING: We see that discussion picking up a bit, maybe we heed to slow down innovation, think what's at stake here. I don't think it's specific to the Red example but we do pick that up there, there is some discussions about how far can you take and raise that air wareness. It comes back to industry and health and safety, make sure this stuff keeps your workers safe because that as a company is your responsibility.
AUDIENCE SPEAKER: Speaking as radio operator, if there are exceptions this is only valid for Hem radio so for nothing else.
MARCO HOGEWONING: Thanks for that clarification.
JIM REID: Thank you, Marco. We are now going to have a presentation from Michael Richardson beaming in from Canada but before we do that, a couple of quick apologies, I skipped over one or two pieces of administrivia, we have the minutes of the last meeting which were circulated, there has been no comments about this on the list so I take it they have been approved, using my power as Working Group chair to unilaterally decide that. There has been some discussion about the fact that this Working Group is clashing with the open source Working Group and that is felt not being an ideal mix. If anybody has got concerns about this could you please come and talk to me during this meeting or comment on the mailing list and we will try and do something about it. If this is not going to be an optimal mix for future plans we will try and clash with some other Working Group instead and annoy a different group of people. We are back on track. And now as if by magic I hope Michael is going to beam in from can at that.
MICHAEL RICHARDSON: I can hear you. So are my slides on your screen?
JIM REID: They are.
MICHAEL RICHARDSON: So I am talking today in place of ‑‑ at OARC meeting but apparently he has gone off to do some kind of vacation thing I don't understand involving his wife. So I will be talking today about a project that he has been leading in and I have been involved in called the home secure gateway. I have got a couple of pretty motherhood slides here about today's networks and how home networks are not that secure, particularly they lack a lot of testing to be secure and they are not typically designed at all. And I am speaking about home networks and also pretty much small business networks that have no really active person running them.
We expect that home networks, they must be safe, must be private, secure and most of all easy to use. If you are familiar with the IETF home networking group we believe that that Working Group is making significant strides in making the home router essentially plug and play and the business of how many you can plug in and what cables go where, becoming kind of really idiot proof and so we that I is a good thing and we think some progress happening there.
IPv6 is going to make the home network reachable and we think that's good, but it must be done in a secure way, and we must have some kind of controls over which things go where, so along with this increased power comes some increased responsibility and ‑‑ provide some of that. They are going to grow and become personal, they are going to become wearable, we talk about IoT, inside the home and outside the home so we see that networks may, extended into people's automobiles, bicycles who knows what else and these devices will travel, we know laptops travel outside the hope, come into the home and sometimes in fact the home because of their travels. So how to we get here? Basically, our assessment of the Maher aye attacks suggest we need to do some preventative things if we want to prevent future denial of service attacks, we heed to be able to basically significantly improve the state of home networks and so we have identified some open source projects, OpenWrt and we have embarked on a journey to identify what we need to change and hopefully better the Internet.
So our goal here for the secure home gateway is to protect the Internet from IoT device attacks, so if you are a TLD operator that may well really matter to you, at the same time to protect the home IoT devices from Internet attacks and even without IPv6 we know that there is a fair bit of accessibility through part opens and insecure clouds and things like that, and so we kind of take it as read that IPv6 is here, devices are addressable even if they don't think they have IPv6 they are still addressable.
What have we done, we have developed a secure gateway home prototype and we are looking at basically what standards do we need to enhance, what standards do we need to implement and finally, since it's CIRA and care about names, how can we name things in the process, as long as we have security the home network and the devices, then we probably can hear the devices names in a useful and intelligent way.
Why are we working on this? So as I said, CIRA observes that large distributed denial of services attacks like the Dyn attack of 18 months ago now, is a most serious risk to TLDs and it's worth a certain amount of prevention in order to deal with this and to keep as it were the weaponisation of IoT devices.
And while individual devices may be relatively low powered and may be even running on batteries in the future, the individually may be unable to threat en, be a real threat, the sheer scale of them makes it a real issue. At the same time, the current crop of so‑called IoT devices that we see, web cameras, servers, teddy bears, whatever, are frequently plugged into the wall, lots of compute power, might have gigabit uplinks and individually can do quite a bit. But light bulbs will be much less powerful but an awful lot more of them.
I have talked about this slide, I will continue on. So one of the thoughts that people have said is don't think about connecting these devices up, it's too taken RUS and the answer is, well, okay some people may decide to do that, but the utility of them is usually because they have connectivity. And so the question is, how much connectivity do they really need? And can we control this with a gateway or firewall?
So the idea is that we will identify in our gateway the different devices that you might have, we are going to split them into different networks, we may not allow the different pieces, the sensor network may not be allowed to talk to the appliance network, the home security network may be prevented from talking to anything other than the home security monitoring company, and there is a fairly, can be a fairly complicated thing and this is usually the ability of most poem's set‑up, if you think about them all being on the same wi‑fi, then it becomes actually a real question as to can I even segregate them at all, and the answer is that you may know that most wi‑fi now all traffic goes through the access point regardless so we can hairpin them and set up policies if only we knew what those policies were.
So, a new specification to the rescue, we have from the IETF Ops area Working Group the manufacture usage description or MUD file, and it is described in YANG implemented in JSON and it is a statement essentially that this device needs or reasonably would need to access the following destinations or may be accessed from the following destinations using the following port numbers. So essentially this is a machine‑readable description of a set of firewall rules and if we can get one of these for every device in the network then we can reasonably set up the right rules and that is what we have set about doing.
There is a bunch of different ways of getting MUD files, the nicest way is if they provide that information through DHCP and LLDP, when they enroll through a protocol called Bruce key ‑‑ they don't implement those protocols yet what if the end user could simply scan a QR code and that is what we have done. And we will implement the other things as they come along and there are some other people have done some DHCP work which we have also incorporated but it's not at this point the primary method.
So how does it work? Basically, we notice there is a new device and don't give it access to the world, we have to give it an address so we can see who it is and we put it in a list and there is an app that we have been writing that runs on iOS and Android and ideally we could make it a one page app for browser. And it talks to your gateway and you get to look at the list of at the vices and you can scan a QR code. The authentication between your smartphone and gateway is not password based, we are using TLS and we have an interesting proposal on how to set up which we think will be fairly effective. One of the things we do is we do go out to our own Cloud repository to get the MUD file and the reason for this is that at this point, we don't know that many vendors had a have MUD files and so we are good at looking at juristics looking at MAC addresses and things like this for finding MUD files that may be in fact Crowd Sourced so we may have a curated database of this this is what we think this device needs, we don't know whether the manufacturers are going to be intelligent to that have have MUD files and overstate what is necessary for the device and we may want to reduce that part a bit. And so the MUD process allows for some of this. The other thing that we have realised is that it's unclear in the MUD files whether or not the access being asked for is regular access or maybe for firmware updates. This needs to be easy to use and we are thinking how can we do this. If the appliance violates the set of lists, the criteria, then essentially we consider them to be a bad at the voice as it were, the refrigerator has gone bad and we are going to it quarantine it. When it's quarantined, won't be able to access any of its normal resources or Internet so one of the things we are looking at is how can we make sure can still access the manufacturer for the purposes of getting firmware updates so that the problem can be fixed? Or the device can be re‑set to manufacture default.
So to make it really easy we are thinking that our user interface is going to be the Tinder of IoT device, swipe left, right, up, and whether you like or hate the device or want to simplify ‑‑ and that what you do for your laptop, you'd say no, it gets access to everything regardless, I don't need to have a MUD file for that. So that is the kind of interface we are looking at doing that we want to make it as simple as possible and we want to make it such that you really could receive a message or your phone that says that your spouse or your child has plugged in a device and you can consider whether or not you are going to allow that to happen that day. So we have implemented this on Omni Turris gateway, we have some code written enin lieu a and ‑‑ we have put together and it's all essentially visible on our GitHub variation of these slides are in URL at the bottom. And I don't have time to give you a demo today but if you want to see one at ICANN this weekend where Joch will be presenting similar things. At this point it is a proof of concept and we have a number of logistical and packaging and other issues that we need to work out but there is a half dozen of them set up and running in our labs at this point.
Some of the specs that we care about that we are working on and if you want to contribute to them, that would be wonderful.
Questions?
JIM REID: Thank you very much, Michael.
(Applause)
AUDIENCE SPEAKER: How much time do we have?
JIM REID: Five minutes.
AUDIENCE SPEAKER: From Qrator Labs. Who is going to implement this?
MICHAEL RICHARDSON: Yes. So that is a very good question. So we think that it is in the interest of ccTLDs to make sure this happens and among those that realise that if they want to prevent weaponised IoT it's necessary to make an investment. And I think that's the discussion of why they were talking with about this at ICANN this weekend. It would be nice to have ISPs committed and interested who are shipping equipment, but I mean, our experience with quite a number of ISPs unfortunately I know who I am talking to in the room here, is it's slap something together and ship it out or buy whatever is cheapest this week and not a lot of responsibility for what comes out of those devices, I know the people in this room care what is going on, I know you will be interested in participating going forward.
AUDIENCE SPEAKER: First of all, you are expected to speak about IoT but you only mention smart homes. So here I hold a report from a reseven by growth enabler, I am not sure if it's available for free but I can show you ‑‑ I cannot because you are in remote ‑‑ anyway, so according to growth enabler smart homes account for only 14% of total IT market, well for example, sorry, smart ‑ account for 26, industrial IoT is 24 and connected health is 20. The funny thing here is that the approach you have proposed might fit well for industrial IoT, connected health because the worst device, embedded devices I have seen in my life were medical devices and we actually have a security issue now implementing QUIC across our network due to those magical devices.
JIM REID: Is there a question coming?
AUDIENCE SPEAKER: Yes. For smart homes take Rumba ‑‑
JIM REID: We are not here to talk. We have a limited time.
MICHAEL RICHARDSON: I think I understand the question, why don't we look at industrial and other things and I think there is a couple of reasons for this. That the first thing is, that the protocols that we are proposing to apply in the home are variations of some of the IETF protocols that are being developed for specifically securing industrial IoT things. So in some sense what we are in fact planning to to is figuring out how to apply the industrial solutions which are managed in typically and have an operator to a place that has no operator and are difficult to manage. Industries that screw up and weaponise their industrial plants into attacking the rest of the Internet I think can be held accountable for a legal point of view and I think that is what is going to have to happen. We can have a conversation about medical stuff. It's a really complicated problem, most of which because ultimately when you come into the emergency, the doctor needs to man and middle all your devices and that's critical to making a proper diagnosis and figuring out what is going on with you so that is a real challenge and I am not going to pretend this is going to help at all.
AUDIENCE SPEAKER: A different challenge is that smart home vendors, I mean they are not going to implement this because it hurts their business, not going to disconnect ‑‑
MICHAEL RICHARDSON: Right. That is why we are going to put a firewall around them and when they misbehave we are going to know and be able to say this particular system is screwing up and we are taking you off‑line and it's going to become apparent how many times each of the very smart homes verticals have screwed up, so I think this is a valuable thing that way to keep them account of what is going on.
JIM REID: Let's continue this discussion on the list. Next question, please.
AUDIENCE SPEAKER: Jelta, semi‑connected breathing thing but I also work at the SIDN and we are working on the spin project and I am sure you are aware of it.
MICHAEL RICHARDSON: Yes.
AUDIENCE SPEAKER: We have quite a few similarities and it's nice to see you are reaching the same conclusions about MUD.
MICHAEL RICHARDSON: It is one of our things on our list to incorporate in the next maybe not in the next phase but the phase after that. So, very please don't stop working on what you are working on, we want to ‑‑
AUDIENCE SPEAKER: Right now, I am working on MUD so we have two competing implementations. What I was thinking, maybe we should standardise that way to spread MUD files because I don't think that the manufacturers will provide any decent MUD specifications. Have you thought about that yet?
MICHAEL RICHARDSON: So our initial thought is we are going to add a Trust Anchor, we are going to man in the middle them, the ones we get from manufacturers and create a curated database of ones that come from the community or changes that come interest the community, has to be curated because we don't want just anything in there but it might be as open as here is a GitHub, send a pull request but we haven't really figured the logistics of that. We just know in addition to trusting well, I say trusting, validating the manufacturer's signature or what might be the signature because MUD isn't very clear about how we get a trust relationship there, we will be validating signature from our own Cloud, from CIRALabs and we may to some things that way. Of course, you know, really sophisticated end users can change the trust anchors if they want or load MUD files their own way but we are not targeting those sophisticated users but rather the average person.
AUDIENCE SPEAKER: It might be worth brainstorming about a more general solution for that.
ROBERT KISTELEKI: Individual citizen. Thank you very much for your working it's very exciting. I wish I can contribute and I will try to see how I can. I will try this to phrase this ‑‑ in order to save time, I wonder what the role of machine learning could be here? Like if the device could ‑‑ not the device, but the firewall can actually learn what normal behaviour is and then just act if currently it's different but as a sidenote this has been one of the best quality remote presentations that I ever had, kudos to whoever made it happen, you are magicians.
(Applause)
JIM REID: Thank you to all the meeting team for making this happen. Thank you to Michael as well.
(Applause)
MICHAEL RICHARDSON: You know me, thank you.
JIM REID: Next up we have two representatives from the Dutch police, Manon den Dunne and Jaap Van Oss, I hope they are going to tell us it's going to be a criminal offence to write IoT crap‑ware.
JAAP VAN OSS: Having 20 minutes to explain to you what the police perspective is on IoT is so we will do that very quickly and without losing any time I give the floor to man on who will start off.
MANON DEN DUNNEN: I work as a specialist on digital transformation, one of my key area of interest is IoT and I am part of the next generation Internet programme of the EU and IoT community in Amsterdam.
So, the Internet of things is really accelerating, the digitalisation of society, which makes it possible for products and services to be better tailored to people's personal circumstances and needs. And thanks to the possibility now to quip the environment objects and even human beings with sensors more and more data is becoming available to even better and improve these personalised treatment in their living environment so people with mobility problem can be recognised by a pedestrian traffic lights, which can stay longer on green, people can stay home longer, we just had a talk about smart health products, air plugs can be cut, safety improved. All examples of services that are now developed in the context of smart cities, smart health. But as we move towards a society or the Internet of things is omnipresent and a distinction between online and off‑line, we are in a situation where 24/7 information is gathered about individuals, the environment but also the context. And even though that can be used to better their personal situation, we also see a dark side, a shadow side to this development, it's unclear not transparent what kind of information is collected. The quality of the information or the data collected, what value is attributed to the data collected and how they are mutually combined into personal profile which decides what kind of services, information and even treatment you get.
This we recognise now, that this can undermine the rule of law. Because in the Netherlands, we have this Constitution and this developments that create new forms of exclusion, reinforce and confirm existing biases and discrimination, think only about the research that I as a woman if I search on Google for a job I get less high paid I don't be offered than a man that's uses the same search query. Or in America now it's dependent on if you buy fennel or not when you go to the supermarket how high your insurance premium will be. Furthermore, I summarise it in this tile. It's unclear what your behaviour will have for consequences in this new world. And that affects our constitutional rights and some of you might think that the police is here to investigate crime, but that's only a means to an end, the police is there to protect our constitutional values. Furthermore, the increasing dependency on devices, infrastructure that is poorly protected facilitates cybercrime about which Jaap will soon talk. This is a major challenge for us how to deal with these aspects in a global connected world. And we were very pleased to learn in 2017 that they had come up with a smart city strategy for the Netherlands and in this strategy, it describes the approach that, the smart cities is to ‑‑ objects of smart cities to it solve societal challenges and that a safe and secure infrastructure is one of the important preconditions to realise that. This is not an easy thing to do, realising a safe, open accessible digital infrastructure because it's not only about physical infrastructure like the cables and the network, you can compare it it with the road, the public road where they are ruling about which cars can enter but can you have a terrorist on the side walks so it's creating a new digital, livable environment for people. And the city of Amsterdam and Eindhoven took over this challenge and came up with four key principles and summarised, it's about safeguarding our critical constitutional values for the future, manufacture siding the importance of openness, operability, and data, that is the last one. I hope you can imagine how revolutionary that is, collect data and try to sell this data as well. The world we are heading to assess that this data is public property. It's all our data. This cannot of course be the single responsibility of a single party so Amsterdam took the initiative to gather a lot of public and semi‑public institutions, to start a programme to translate the intention of this ambition into building blocks and principles a realisation and I think, well I am proud that as the police we are co‑founding this programme. So what we have done until now is realising this mission by, for example, creating IoT registers so there are a lot of cities in the Netherlands now creating an IoT registry where everyone can see what kind of data is gathered and collected about them in public space and we are working on trust framework and a trust framework is meant to it create new rules of engagement by which at national level we decide on building blocks that will safeguard, that you can organise on the one hand the access to your data and devices as to allow to share ‑‑ you to share them with others and on the other hand, to give conditional access to data for governments, companies and civilians who want to use that data.
This is work in progress so what we have done, we have decided on several building blocks that need to be, that cannot be in private hands, that we have to facilitate publicly. Like the Internet. And several building blocks are already available like the block chain or IRMA, I hope you know that app, I reveal my attributes, you don't have to share the data itself but you can share the necessary attributes and identification tools like DGT. No data is stored in this layer, the only about agreements, conditions for sharing data on devices. There is not enough time to dive deep. I am sharing this with you to get some feedback, I also got some good ideas from the previous speakers but I want to give the floor to Jaap now to tell a bit about the background of our cybercrime perspective.
JAAP VAN OSS: Thank you very much. For a moment I was scared that I was only eye canned eon this stage but I still have a couple of minutes. What we hear today, I heard Red, I heard MUD and webising IoT for denial of services attacks and that is the same with me, when I think IoT as an investigator I think MIRAI, I think denial of service attacks. We know them, website, Dyn attack, even Deutsche Telekom got hit badly. You may think it's old news, it's not. The majority of denial of service attacks is still IoT botnets.
But first, January, this came into the news and the Dutch intelligence and security services allegedly had insight into the hacking group that was interfering with the elections in the US. And that became really big news. I can't imagine that. But normally our intelligence and security service never gives this information away. Nevertheless, what happened was in the weekend thereafter, our banks got hit and ‑‑ the weekend thereafter the banks got hit, and immediately it was thought that must be retaliation from the Russians because our intelligence security service were there, nothing of that was true. There was no connection between the events in Russia and the denial of service attacks that weekend. It was basically what is bothering us as a society, it's like relatively young kids use cheap means and facilitations to execute denial and service attacks. That is what is happening. And the majority of those attacks are still relatively innocent and I say relatively because the banks go down and so it's relative impact. But it's not state exploited, it's not a terrorist attack yet. Nevertheless, that is the fear of course. Okay.
So we do realise that DDoS is a problem, on the other hand it is very hard to investigate. We all know that. And that is also the problem with these IoT devices when they are not secure, you know, wherever this attack is coming from you will never know. So, what we to in the Netherlands is basically try to stop denial of service attacks from a different perspective, we try to, yes, to investigate, but we also try to look at the big facilitators of these denial of service attacks and try to look at alternative interventions and try to see how we can make this IoT devices more secure because that's what our, what keeps our society secure.
Okay. Of corks we investigate and we take down sites like the web stressor one, we still do that and we probably want to take down a couple more of them and it's also showing that there is a reprecious action possibly when you deal with IoT attacks but just top of the mountain. What we are also looking at is if we can somehow derive signatures from those attacks, so we are building our database with attacks right now. So that if a bank or an another organisation or one of our governmental website is under attack that we can compare the signatures of these attacks and that gives us a lot more possibilities to investigate than we had so far. So, it's, yes, repression but also thinking new ways new instruments. This means, of course, that you will have to cooperate, so we are not alone in this society and we are really thinking about how to basically implement a much more secure programme against attacks with IoT devices and, for example, DDoS attacks.
So we might improve our information position as police but also throughout the Netherlands itself, we want to somehow provide better tracking and tracing systems, we want to ‑‑ we also want to attribute to the criminals and the attacks and basically want to be able to somehow defend ourselves properly against these attacks.
This police initiative is called the no more DDoS, it's some kind of initiative but completely different. This is to show police is not only knocking on doors and trying to grab everybody, we do have responsibility there and we feel it's a shared responsibility. That's why we want to be here and also, you know, position ourselves as relatively nice cops that do feel that some things should be done.
So I think our message is quite clear and what we want to put up in this discourse, in this discussion is that how we can fulfil from our end the part of the shared responsibility to make these things more secure and specifically with looking at the digital public space and also in fighting cybercrime we know there is a shared responsibility so that's basically what we put up here on this table and I think that's where we end our talk and have one minute left for questions.
(Applause)
JIM REID: If Manon and Jaap are good cops, does that make me the bad one? This should be start of an ongoing dialogue, very useful content and this is very thought provoking, butth both what you are talking about in terms of this whole framework around agreements and how to share data and how not to, I think that is very interesting and that's certainly worth further discussion and consideration here and I hope very much we can to that. As to you as well with Jaap, obviously the ongoing operational issues here are something very dear to many, many people in this community and I am sure there will be further cooperation here, we have a question but please be brief and direct.
AUDIENCE SPEAKER: You have said that the majority of the DDoS attacks comes from IT devices, what is the source of this data? You have said that the majority of DDoS attacks comes from the IoT devices. What is the sort of this information?
JAAP VAN OSS: Just my general knowledge in investigating these kind of crimes. I do not have it completely counted like you do in your figures, but I just want you to know that, you know, these MIRAI‑driven DDoS attacks do constitute quite a big number in the overall denial of service attacks.
AUDIENCE SPEAKER: Fair enough. Thank you.
JIM REID: Thank you. Thomas is going to talk about the security of IT devices.
THOMAS STOLS: Hello everyone. So real brief about me, so I am a security consultant/hacker whatever name you want to gave me, I work for a company called Computest, we are a digital security company, my background is in programming and I am a member of the IoT team which means that I get to test a lot of IoT consumer products, and I want to talk to you, I want to share a bit with you the findings we have mainly in consumer products.
So we at Computest, besides our commercial activities we often do research mainly in software project, these are some of the vulnerabilities that we have discovered recently and so I'm also part of some of this, these research projects.
I think it was mark or Marco, right, who mentioned Bruce Schneier already so in the need for interconnectivity is increasing, more and more devices have network connectivity and network devices are getting more complex. In 1999 it was Bruce, one of the security gurus who said there were little incentive for spending more time and money on a secure product, and as a result, low quality and little secure products are entering the market at a high speed. And Bruce Schneier states unless customers demand higher quality and better security this was never change. This was in 1999 and to put it in perspective that was the year of I mod, it was the year that Internet explorer 5 was released so this is a long time ago. 1999. And then he said well I see two alternatives, either we can keep on going like headless chickens and keep this in the back of our minds like we know this, we accept the risk, or we can slow down reevaluate and start working on how we are going to secure our products. This was 1999 and as we all know we chose option one which has kept running like headless chickens so by now we have 220 times more connected devices than in 1999 and the problems, the security problems still haven't gone away.
Are we in worse shape than we were 20 years ago? Well, let's see. We tried to everything nowadays. This is a water bottle that is connected to the Internet somehow as an app that registers your daily water intake. These are hair brushes that coach you while you are brushing your hair, it measures the amount of force that you apply to the scalp, it tells you how frizzy your hair is, that is exactly what we want. And this is a smart toilet, right? This comes with an app that can personalise your toilet experience. So, I can imagine there is some spraying going on and that you can adjust the settings a little bit and things like that. It seems like we just connect products because we can, and maybe we should stop and think if we actually should. Because there are security risks there. This is a fit bit bathroom scale which comes with a mobile app which is connected to the Internet. This was vulnerable to remote code execution, even though the actual device itself was on the home network, so by somehow connecting to the Cloud platform itself it was possible to gain remote route access to this device, which is actually a great entry point for a hacker to your local network, just by hacking this device. There was recently was a story about a casino getting hacked by their IoT thermometer which was in an aquarium somewhere, usually have good digital security as well but this was an IoT thermometer which was vulnerable so hackers that were nearby I think, they hacked this thermometer and then they obtained route access and used that as an entry point into the casino network and took out all the high roller gambler data.
We have seen published vulnerabilities in medical devices as well, some people talking about that before me. This is a wi‑fi enabled insulin pump, controls the amount that a diabetic patient receives, a vulnerability was found that allows a nearby attacker to overdose a patient. So this is pretty serious stuff.
Also some inventive groups so in 2016 a research firm called Med Sec found vulnerabilities in a pacemaker and allowed them to remotely control the pacemaker. But there was no bug bounty for this manufacturer so they couldn't get any money for their research so what they did is they partnered up with an investment if I were, shorted the stock of St. Jude medical and published their research. And they earned a lot of money through that and it's kind of a grey area; I mean, should that be ethically ‑‑ is that okay? I am not sure. If you don't run a bug bounty programme and you are running these devices you should reconsider or do some disclosure policy of some kind.
Other risks: If you want to update such a device, because it's vulnerable, that could essentially mean surgery, because it might not be remotely updatable.
I want doing over some of the main problems that we are facing in the IoT industry mainly in the consumer product area. The first thing is cost. For most IoT devices cost are the main business driver. Consumers often choose the cheapest product that offers the most functionality that they actually want to use. And to reduce cost, devices are low on resources, there is no room to invest in security and that's okay because security is not required, it's not a requirement. And we see this in routers, right? There has been several publications about how easy it is to determine the pre configured wi‑fi passwords of routers if you know the SSID or MAC address of the router. This is mainly because there is an algorithm in there that based on the SS IT or the Mac address just generates one of these default passwords. So instead, what we could do is spend some extra money on he separate labels and generate random initial passwords. But again, it costs the manufacturer more money. We have insecure communication so there are some popular protocols being used in the IoT world which are like web sockets, M UTT, we have basic rest APIs with or without ‑‑ we have several of these protocols and most of these have no authentication or encryption by default. So, in some platforms encryption and security is not used at all, and we know that using asymmetric cryptology and a proper PKI infrastructure is the right way to set up an IoT platform but there is no actual requirement in there. So, a lot of these companies, manufacturers, they don't do it. As an example, I once had to test one of these access systems so this is a door lock with a pin code and a fingerprint and this device, it didn't check the certificate of the actual Cloud platform it connected to, so what I did is what ‑‑ I intercepted the network traffic, I offered the device a fake certificate and then I was able to see and observe what kind of network traffic is transmitted by the device and basically what it did is just posted something to an end‑point with its serial number and action. From that moment on it was pretty obvious what I could do, serial number by the way was on the side of the device so you could just read it and post to a specific end point at user Thomas with PIN 1234 and I was able to enter the building with 1234, it was that easy.
Update mechanism, some devices cannot updated at all, some devices can only be updated if a user initiates it. Even Windows, which has existed since 1985 still needs to patch security flaws every week, and that's okay. I mean, having vulnerabilities is okay. That comes with building software. However, it becomes a problem if you cannot fix and update them. In mobile device the update processes have matured. Right? They are all automatic unless you specifically tell them to be manual, in IoT this transition is still ongoing, we still see a lot of manual updating there.
And an IoT products often need to last for a long, long time. If we take a car which can essentially be considered an IoT product right, that needs to be on the road for 15 years, so, this also means that vendors have to provide support for that amount of time, and not only the vendors but also their hardware suppliers need to provide support for that amount of time. And at the moment manufacturers are just not equipped to cope with this. Extended support is not included in their margins, in their costs, it's not included at all. So if they need to extend their support, the prices of these devices will have doing up. IoT platforms are also increasingly becoming more complex so an average product is overwhelmed with sensors and features and things and again I am taking the car as an example because we did some research into car vulnerabilities and we were able to remotely own a system of a Volkswagen group car but a car contains 20 to 50 kilos of cable, over 100 computers which are all interconnected, over 100 million lines of code is used in the software in one of these modern cars. And all systems are interconnected by means of a can bus and this standard bus, it connects the engine, the brakes, the Windows and the locks and everything. So, basically, from an external perspective, this is the attack service, and if I am able to obtain access via any of these means I could essentially break the car, I could essentially control the engine. The car manufacturers have come up with solutions for that which is basically a gateway system between the essential systems and the non‑essential systems which is a can bus gateway and you cannot in some cars, you cannot access the engine from the infotainment system any more, except for a jeep which was in the news a couple of years ago, this was the architecture of a jeep, it was all interjected. So by abusing a vulnerability in the infotainment system, researchers were able to stop the car.
Consumers are often responsible for security and I am talking about hooking up IoT devices to the Internet directly by mistake or purposefully, and not changing default passwords. [M IRA I] has been mentioned a couple of times so that painfully exposed the fact that many of these IoT devices still have their default passwords set. This led to one of the largest distributed denial of service attacks in 2016 I think. And because a lot of these IoT consumer products they run Linux, we also see that there is a major increase in the volume of malware developed for Linux, since IoT products are a nice target.
As a consumer, so this is a very funny, these are baby monitors, I made this picture myself so I was looking for a baby monitor and I was in an electronic store and I saw these and I thought to myself how do I know what baby monitor is secure? I don't. As a consumer, there is no insight into what product is secure, even though there is a price range from €10 to €100, I don't know what is secure and what is not. And that camera is pointing at my baby all day long. So, yeah.
And what we also see is a knowledge gap so developers that make web applications, they have learned a long time ago to defend themselves against the dangers of the Internet. They are used to their application being accessible over the Internet. Well, in the IoT world we see that embedded device programmers are now this. They don't understand this Internet thing yet. And their security development practice, they were never a priority so there is a knowledge gap there.
And we saw all of these things before when we started building websites and applications pour the Internet. In the application security field we have solved many of these problems already. So maybe we can learn some things from the application security field. For instance, we know that users are bad at picking passwords so Apple has been very innovative in trying to come up with new ways for authentication, for instance using your fingerprints or your face. The solution often is not security awareness training but just making stuff easier and making stuff easier helps. And we also see multi factor authentication becoming more and more by default and also more and more simple where you used to need to fill in an actual PIN code or generate a code and type it in, nowadays you can get a pop‑up, a push message and you say allow and it works. So, multi factor authentication has also been made simpler, easier and more secure. And vendors auto update their software without us knowing it; I mean, these are practices that we can take to the IoT world as well. And actually, Tesla was recently in the news where they introduced a PIN code before, that you can activate so that before you start a car you need to actually type in the pin code before you can drive away and this is a solution to the key fobbing problem where criminal are actually using these antennas to unlock your car.
So are we doomed? Well, I think not. There is some great initiatives so the big Cloud providers, they are all committed ‑‑ committing themselves to IoT, providers also start to supply hardware components in addition to these Cloud platforms. So, and there is dedicated security teams working around the clock to solve these issues. So, are we doomed? I don't think so. But we do have a long way doing. Thanks.
(Applause)
JIM REID: Thank you very much. We are running a bit over time here, I hope the Working Group won't mind if present all their ideas I would ask people who have got questions to please keep them brief and direct.
JAN ZORZ: Thank you for this. So do you consider if an IoT device like a toilet, for example, or this device for diabetics has the smart in the core and connected to the core and can be controlled from the Cloud, is this considered as a security risk or not?
THOMAS STOLS: Not if you properly secure the Cloud.
JAN ZORZ: Good luck with that.
THOMAS STOLS: Of course there is always a risk. However, I guess you have to figure out whether you want to reduce ‑‑ whether you can reduce that risk as low as possible so that you can accept it.
BRIAN NISBET: HEAnet. I realise this wasn't the core part of your presentation, but I have got to ask how we can even begin to consider shorting the stock on a manufacturer just because they don't have a bug bounty programme, to be within anywhere near ethical. I mean it's bad and wrong and I am not so worried about the executives of the company but that is going to lose people jobs and cause massive problems so can we not just say that these kind of things are massively unethical.
THOMAS STOLS: I fully agree with you, only the for the law it's a grey area. These people, I don't think they have been prosecuted and I don't think they will be for this. But again I agree, fully agree with you it's ‑‑
BRIAN NISBET: There is some law enforcement behind me they can do something.
MANON DEN DUNNEN: You mentioned Apple introduce fingerprint and facial recognition, the facial recognition has been fooled, so I wonder do you really think that share your biometrical identity is the way doing considering that after it's been hacked you cannot re‑set your face?
THOMAS STOLS: Of course. There should be considerations there. I mean, maybe you can add this as an additional security measure, maybe you can figure out a way to make the re‑setting process a little easier.
MANON DEN DUNNEN: But it's giving away the closest things to your identity. Sharing it on the Internet.
THOMAS STOLS: Yes. So that is a risk you have to be willing to accept. But I think usability‑wise it has improved the way that we authenticate. As opposed to passwords.
JIM REID: Thank you very much, Thomas.
AUDIENCE SPEAKER: Peter stein houser, just a short comment on the 3D facial recognition of Apple, it's not going to the Cloud, it stays in the device. Another short note: As far as I know the state of California made a ruling that every device sold needs a unique password, which is actually good but the bad thing is they may made not any recommendations how these passwords should look like. So, I don't know if it's making things better or worse. Thanks.
THOMAS STOLS: Thanks.
AUDIENCE SPEAKER: From cybercrime of Greece. I know that it's only comment, and I think right now the European Commission with ‑‑ coupled by European agency for information security and they are trying to issue some kind of cybersecurity certifications for all these products I think. And it's something ‑‑
THOMAS STOLS: I see a lot of initiatives by a lot of governmental institutions that are working on this. But we haven't found the solution yet.
JIM REID: Thank you. And also points out there was something posted on the mailing list the other day that the UK government has announced some guidelines what should be required for devices. So thank you very much Thomas. And we will move on to final speaker of the day, we have got Arman Noroozian and he is going to be talking about IoT security.
ARMAN NOROOZIAN: Good afternoon everyone. I will try to keep my talk a bit short, I don't want to keep you from whatever is coming up next, I can skip a lot of my slides because there were interesting examples that Thomas gave before and all the previous speakers, I am from inter disciplinary research group and most of what I am presenting here are work of my colleagues on their behalf and what I am going to present a bit is research into causes and trends and some responses that we can have to security of IoT devices. But it's going to be slightly from a governance perspective.
So, I am going to just skip all the known examples, we know the disasters that things like /PHEUFPLT IRA I can cause and we also know projections shows that IoT devices are flooding the market and they have a lot of security problems. But that kind of raises two governance questions which is like how is it we get into this mess and how to we get out of it? And just to take a quick look at the first one so there is a fragmented landscape of all manufacturers and vendors producing IoT devices and a lot of them are incompetent and don't have the right incentives to invest in a security as the previous speakers talked about. We have a lack of visibility into the devices that failure, so many devices out there we cannot simply practically measure and figure out how many are vulnerable. It's just everywhere. And again, to take the example that the previous speaker said, there are dependencies between car manufacturer depends on a lot of components that he gets from other companies and these are interconnected in very complex ways so you cannot try to pull out devices from the market or to some other things.
From the governance perspective how to we get out of this? The situation is trying to mop a floor while the water tap is still running, that is a difficult problem to try to solve. I will quickly overview some of the approaches that were ‑‑ that are being discussed, for example a lot of these were discussed a few weeks ago, there are things about raising awareness and it's important to not blame victims here, the for example the consumers that plug in IoT device they are partly responsibility and needs to be monitoring and transparency about the defenders and the manufacturing ‑‑ vendors and manufacturers so carrots and sticks and naming shaming or praising the good once, there are liability issues, there are for example information like opt‑in mechanisms to allow users to share data or giving them ‑‑ rather than giving them specific rights to return bad devices but then the thing I want to focus on here and I quickly skipped over what the role of intermediaries is, like network operators and things like ISPs can be in this space. So I am going to present to you research that is kind of focused a bit on this issue and for that I am going to first show you some measurements that we is it and where most of the hacked devices are located.
And to do that, we have been operating a honey pot system for several years now with our colleagues in Japan, there is lots of emulated and physical devices actually connected to this honey pot, lots of ports open and a lot of the interactions are being monitored like the standard honey pot set of systems that you can think of. And when we look at, for example, /PHAO*EUR infections, try to see where they are located. So if you look at the infections you see the actual compromised IoT device is most of the time located in ISP broadband network, the also similarly there the bits spread out in other types of networks and C2 component is mostly on hoster provider's network so what to we get from this picture? It's that ISP broadband networks are actually in a unique position to try to defend the rest of the Internet from the negative externalities of infected devices, and if you look at the trends in terms of where the bigger ‑‑ biggest problems are you can see there is this clear trend where ISPs that have the biggest number of subscribers and most number of infected devices. If you look at the same ‑‑ situation, for example, zoomed in in the Netherlands you see the same trend where all of the ‑‑ the majority of the infected devices again are in ISP broadband networks. So, this brings me to the experiment that I want to talk about a bit. And I am going over everything a bit fast. This is an experiment that was done with KPN, that is one of the ISP providers here in the Netherlands and what they have is they have this old tool from ‑‑ within their arsenal which is a walled garden, a quarantined area where a customer is infected or some device is infected, within their home network they are going to can he tell off their access. There are issues with that and there is this paper that is kind of mentioned at the bottom where most of the research is described and it's their work that I am presenting. So, there was a research into basically how effective a walled garden can be in already protecting what we have and we can think about all the other solutions related to IoT problems a bit later. So there were around 1,700 different quarantining actions related to about 1,200 different customers and the first study looked into was this actually walled guard enenvironment effective and what was seen over several months of data collection, this was going on as the ISP was operating, was that about 50% of the customers that went into the walled garden cleaned up their devices and most quarantine only once, you can read the details in the paper, some of the customers had, all of the customers that entered had twice the option to just click out and release themselves from quarantine. The third time required actually some interaction with some of the operators. You can read the details in the paper. There is also table there that discusses a bit conversation with the actual customers that went into the walled garden and some of the issues related to that. But so this kind of shows you that it may have an effect but if you want to show that it has you have to do a randomised controlled experiment and within this next experiment what was done was 220 different customers infected, they were randomly assigned to three different groups, some of them went into walled garden and some of them just got an e‑mail notification and some of them went into control group and nothing happened to them. And throughout this experiment what came out was that actually the customers that went into the walled garden 92% of them cleaned the IoT devices within 14 days. Compare that to, for example, just receiving an e‑mail notification you can see that the lower the curve in you have here the better the thing is. It's showing you how much faster the clean‑up was observed. So e‑mail was sitting above the two curves that indicate walled garden set up and the control is basically just above them. So control and e‑mail are kind of similar but we can clearly see an improved situation by putting customers that are infected into a walled garden environment. So basically all this talk has been trying to persuade some of the network operators out there there are already tools that can be used around all of these different parallel solutions that are being discussed in the IoT section and with that, some future research that we are looking at, we have funding to continue this research for about four years from SIDN labs, the Dutch basically scientific organisation for research and the DHS and some of the things that we are trying to do is to kind of take this a bit further and look into for example whether we can using DNS and other types of tools identify the types of devices being infected, where they are located and try to help ISPs provide more useful information to the customer when they get into the quarantine area, objection oh this is probably your camera infected so you can take an action about that. These are some of the directions that we are going to take. With that, I will try to keep it short and thank you for your attention.
JIM REID: Many thanks. I think this work is fantastic and the fact you have some got experiments and hard data hopefully that will be able to persuade ISPs about the control measures they can put in place and how effective they are likely to be, that is great and I want to see more of this kind of thing. We have a question.
AUDIENCE SPEAKER: SIDN Labs. Can you go back to the slide ‑‑ the previous one, where the cleaned up infections. Does this say although insignificant difference but doing nothing is better than just informing your customers by e‑mail?
ARMAN NOROOZIAN: It's kind of well‑known that a lot of ‑‑ unfortunately, a lot of abuse notifications, they either don't end up in the right place or get ignored or go into this kind of automated system that you don't know what happens so a lot of times with our own experiments what we end up seeing is for example 30 to 40% of our emails that we send out just bounce back, they never receive the intended abuse e‑mail and there are all kinds of weird issues going around with that. But I don't know exactly how to read into this kind of thing that you see there, kind of the same. But it's kind of ‑‑ in the research community it's seen as abuse emails sending abuse emails doesn't work.
AUDIENCE SPEAKER: I think we discussed this already but I think it's very interesting the clean‑up rates when you do nothing. That is quite a few devices that get cleaned up automatically somehow.
ARMAN NOROOZIAN: Yes.
SPEAKER:
AUDIENCE SPEAKER: Have you shown this information to the abuse information exchange because they kind of trade in this ‑‑ these notifications and hence in quarantine customers?
ARMAN NOROOZIAN: I am not aware of that detail. I think I would have to talk to the actual researchers that did the work.
AUDIENCE SPEAKER: KPN. Thank you.
JIM REID: Thank you very much. And thank you for saving us a few minutes of time. Thank you.
(Applause)
JIM REID: With that we are just about done. But before we go I have got one short announcement to make. We still don't quite yet have an agreed selection process for appointing a co‑chair. I think we are just about there. I am inclined to declare consensus on the text that has been proposed. There is a small tweak that needs to be made that Peter made, the language is a bit clumsy I am going to declare this done and hopefully we can appoint a co‑chair in time for next meeting in Reykjavik, please come to me if you are interested, and I will be very happy to see any potential volunteers step forward when we actually activate the selection process. Just to say thank you very much for attending. Before we go, thanks as always to the speakers, because without their content and I think we have had great talks today, we wouldn't have this Working Group. Thanks all to the great amount of support we have had from the NCC, the meeting tech time have done a great job with Michael's presentation and also to the Jabber room and the stenographer.
(Applause)
LIVE CAPTIONING BY AOIFE DOWNES, RPR
DUBLIN, IRELAND
