On a previous blogpost we looked at the state of security in Switzerland from an IP-Infrastructure perspective. This time our CEO was invited to speak at Web Zürich, and we decided to test out some new modules that we added to our scanning platform 40fy.
The new additions to the platform now allow us to craft custom requests using domains.
A request will usually look like this:
curl https://api.binaryedge.io/v1/tasks -d '{"type":"grab", "description": "Test Description", "options":[{"targets":["Google.ch"], "ports":[{"port":"80", "config":{"http_path":"/"},"modules": ["http"]}]}]}' -H "X-Token:AUTH-TOKEN-GOES-HERE"
And the response:
{"origin":{"job_id":"3a90f8f1-2180-4c9d-8f21-c7b3fd18d103","country":"uk","type":"job_status","module":"grabber_task","ts":1482153606719},"status":{"started":true,"completed":null,"revoked":false,"success":null}}
{"origin":{"type":"http","job_id":"3a90f8f1-2180-4c9d-8f21-c7b3fd18d103","module":"grabber","country":"uk","ts":1482153607690},"target":{"ip":"binaryedge.io","port":80},"result":{"data":{"request":{"url":"http://[binaryedge.io]/","headers":{"User-Agent":"Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36"}},"response":{"httpVersion":"1.1","statusCode":200,"statusMessage":"OK","headers":{"date":"Mon, 19 Dec 2016 13:20:07 GMT","content-type":"text/html; charset=utf-8","transfer-encoding":"chunked","connection":"close","set-cookie":["__cfduid=d9055e9c05ea21e85f7c164f54010ec121482153607; expires=Tue, 19-Dec-17 13:20:07 GMT; path=/; domain=.binaryedge.io; HttpOnly"],"last-modified":"Mon, 12 Dec 2016 16:58:59 GMT","status":"200 OK","x-content-digest":"11929feada2566fd6584cec1eda53b6ffae9be1d","x-content-type-options":"nosniff, nosniff","x-rack-cache":"stale, valid, store","x-request-id":"fe649595-14e1-4d40-8413-b57c705a5461","x-runtime":"0.495313","x-xss-protection":"1; mode=block, 1; mode=block;","vary":"Accept-Encoding","expires":"Mon, 19 Dec 2016 13:20:07 GMT","cache-control":"max-age=0, no-cache, no-store","pragma":"no-cache","strict-transport-security":"max-age=7776000;","x-frame-options":"SAMEORIGIN","server":"cloudflare-nginx","cf-ray":"313b266efd42135f-LHR"},"href":"https://www.binaryedge.io/","redirects":[{"statusCode":302,"redirectUri":"https://www.binaryedge.io/"}],"body":"<!-- Powered by Strikingly.com 4 (1) Dec 19, 2016 at 01:32--> <!DOCTYPE html> <html itemscope itemtype='http://schema.org/WebPage' lang='en' xmlns:fb='https://www.facebook.com/2008/fbml' xmlns:og='http://ogp.me/ns#' xmlns='http://www.w3.org/1999/xhtml'> <head> <title>BinaryEdge</title> ... </html> <!-- Powered by Strikingly.com 4 (1) Dec 19, 2016 at 01:32-->"}}}}
{"origin":{"job_id":"eb93b9ba-d7fb-4652-9f3c-6022285ebd60","country":"N/A","type":"job_status","module":"job_manager","ts":1482153583150},"status":{"started":null,"completed":null,"revoked":null,"success":null}}
For our talk at Web Zürich the only important part of the response is this :
{
"origin": {
"type": "http",
"job_id": "3a90f8f1-2180-4c9d-8f21-c7b3fd18d103",
"module": "grabber",
"country": "uk",
"ts": 1482153607690
},
"target": {
"ip": "binaryedge.io",
"port": 80
},
"result": {
"data": {
"request": {
"url": "http://[binaryedge.io]/",
"headers": {
"User-Agent": "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36"
}
},
"response": {
"httpVersion": "1.1",
"statusCode": 200,
"statusMessage": "OK",
"headers": {
"date": "Mon, 19 Dec 2016 13:20:07 GMT",
"content-type": "text/html; charset=utf-8",
"transfer-encoding": "chunked",
"connection": "close",
"set-cookie": [
"__cfduid=d9055e9c05ea21e85f7c164f54010ec121482153607; expires=Tue, 19-Dec-17 13:20:07 GMT; path=/; domain=.binaryedge.io; HttpOnly"
],
"last-modified": "Mon, 12 Dec 2016 16:58:59 GMT",
"status": "200 OK",
"x-content-digest": "11929feada2566fd6584cec1eda53b6ffae9be1d",
"x-content-type-options": "nosniff, nosniff",
"x-rack-cache": "stale, valid, store",
"x-request-id": "fe649595-14e1-4d40-8413-b57c705a5461",
"x-runtime": "0.495313",
"x-xss-protection": "1; mode=block, 1; mode=block;",
"vary": "Accept-Encoding",
"expires": "Mon, 19 Dec 2016 13:20:07 GMT",
"cache-control": "max-age=0, no-cache, no-store",
"pragma": "no-cache",
"strict-transport-security": "max-age=7776000;",
"x-frame-options": "SAMEORIGIN",
"server": "cloudflare-nginx",
"cf-ray": "313b266efd42135f-LHR"
},
"href": "https://www.binaryedge.io/",
"redirects": [
{
"statusCode": 302,
"redirectUri": "https://www.binaryedge.io/"
}
],
"body": "<!-- Powered by Strikingly.com 4 (1) Dec 19, 2016 at 01:32-->
<!DOCTYPE html> <html itemscope itemtype='http://schema.org/WebPage' lang='en' xmlns:fb='https://www.facebook.com/2008/fbml' xmlns:og='http://ogp.me/ns#' xmlns='http://www.w3.org/1999/xhtml'> <head> <title>BinaryEdge</title> ... </html> <!-- Powered by Strikingly.com 4 (1) Dec 19, 2016 at 01:32-->"
}
}
}
}
We thought about what would be the lowest bar in terms of security settings we could look at, many would probably target if a company has an SSL version of their website or not, but in this case we thought ever simpler would be to simply check which security-headers are being used on the website. (Maybe we will do the SSL scan on a 2.0 version of this talk as well).
As you can imagine we can do this at a really high scale, for this talk we focused on 1.5 Million Swiss domains, we then looked at the main websites of Banks, Insurances and Pharma companies.
We then ended by looking at the most common websites for doctors here in Switzerland.
You can view our slides here:
And the video of the talk here:
F.A.Q.:
Q: How can I check if some website I'm using has security headers?
A: Go to https://securityheaders.io/ and put the URL there.
Q: What if I want to check all the websites of my organization, we've got thousands of website?
A: Contact us on [email protected], our platform is used exactly for this on a regular basis by security engineers, system administrators and technical staff to debug and monitor these situations.
Q: Does that mean by money,data, whatever is in danger on those websites you talked about?
A: Not directly. We noticed some really weird stuff done on some websites where you would visit the homepage in HTTPS and get redirected to HTTP and that is really bad choice. Our point is that, if something as simple to implement as security headers and that some of them wouldn't even have impact on clients are not implemented, then we should really think about what is happening on the backend where more "complicated things" like data encryption should be done...