API updated with new type responses

Image description

Today we've launched an update to our v2 API which includes new type responses including Residential, Wireless, Hosting and Business. These new type responses are to help you create better custom rules that target specific connection types beyond just the ones we've determined to be proxies or virtual private networks.

We'd like to focus just on one of those new types for a moment, it has been a long requested feature that we identify wireless connections because many customers have problems with malicious users who utilise wireless access points to get around IP bans. This has been because it's very easy to acquire a wireless connection and it's even easier to get a randomised IP Address from a cellular network provider.

And so we've added that type response alongside residential, hosting and business. Like all our types you shouldn't expect a type response to always be there as we won't show you any kind of generic or default response if we don't actually know what kind of a connection an IP is utilising.

In addition to this change we've also made a lot of back-end changes to how the API functions, specifically around storing and retrieving data (which includes our new type information even for clean addresses and location data) and also how the API is initiated on our webserver including API sub-versioning which isn't yet exposed to customers but we are using it internally and it is our intention to make it available to customers in some manner in the future.

So as of right now the new types are live which means you can build custom rules within your dashboard to utilise them immediatly. We've already updated our API documentation ahead of todays launch as-well including the test console where you can try it out in your web browser.

Thanks for reading and happy querying!

New design language & forthcoming API improvements!

Image description

If you visit the homepage today you may notice it looks a lot different to the one we've been serving for almost two years now and that's because we felt it was time to iterate on our design language and follow some of the newer things we've been doing particularly in the customer dashboard.

We want things to not just look nice but be consistent so that all of our pages and the features found on them feel part of the same product. We decided to start with the homepage as it may be considered the most important page of a website when viewed by potential customers.

So what have we changed? Firstly we've changed our call-to-action section which draws your eye to use a wider but softer drop shadow and increased the radius on border edges. Both of these changes give the homepage a more pleasing and modern appearance.

In addition to that we've done away with needless single-pixel borders around some of the information we showed instead embracing white-space and using soft pastel backgrounds beneath important information such as our API URL and our Live API Result area.

But it isn't only the aesthetics we've improved as the live api area now includes buttons for some example addresses and the API URL now updates in real-time when you change the address you're checking. We feel both changes will help to make a much better first impression.

As we mentioned this is simply the first page of the site that is receiving our updated design language and it will be implemented on other pages soon too so make sure to check back for that!

Image description

The other thing we wanted to talk about is a forthcoming improvement to the API, a huge change in-fact. Later this month we'll be enabling a new range of type responses for clean addresses so you can determine if an IP Address belongs to a residential address, business, wireless operator or hosting provider.

These new type responses have been on our roadmap for a very long time and it has taken considerable effort to provide this data with a high enough confidence level that it's not just usable but reliable. We first began writing code to implement this feature a little under a year ago and it has taken until now to reach a point where we feel it's ready.

But it's not just about the data itself as whilst we prepared to deliver this feature we went through the API and overhauled some of the ways we store and access metadata about all addresses (ISP, Location, Type, Threat etc). This was necessary to make available the new connection type information for all addresses, not just the bad ones. The resulting improved code will help us to deliver new kinds of metadata and in a more timely fashion in the future which is something customers are always asking us for.

The new clean type responses once available through the updated v2 API will be accessible like all our features to all customers whether you're on a free or paid plan. That means we'll be one of the only API's in the world offering location, provider, connection and anonymity information about IP Addresses for free. This is something we're unequivocal about, free matters and we are committed to our full-featured free offering.

That's everything we wanted to share with you today, please check back soon for another blog post where the new clean type feature will be going live. Thanks for reading and have a great week!

Improving Account Security

Image description

Last month we made a post where we told you about a new feature within the dashboard that rewards you with extra custom rules when you secure your account with a password and a two-factor authenticator.

Today we're bolstering your account security in two major ways to help combat account takeovers which have been steadily rising over the past year.

The first change is if you have a two-factor authenticator attached to your account then you're no longer able to create an account recovery code through our automated process here.

Instead when using the account recovery page you'll be sent an email where a support representative from proxycheck.io will accept evidence of account ownership from you and the recovery request will be manually evaluated. This is to stop situations where someone compromises your email account and then has the ability to gain access to your proxycheck account through that chain of access.

The second change we've implemented is login security alerts. From now on when you login to the dashboard using an address we've not seen you use before we will send you an email detailing that login so you can quickly take action if it wasn't performed by you.

As we mentioned at the start of this post account takeovers are on the rise. As our service becomes more popular so do the attempts on your accounts. We've seen a large increase in credential stuffing and so it's very important that you secure your accounts, we really cannot stress that enough.

Thanks for reading and we hope everyone is having a great week.

Switching email providers from Mailgun to Amazon SES

Image description

Today we've made a major change to the way we send email by moving from Mailgun to Amazon SES (Simple Email Service) and we wanted to explain why we've made this change and discuss some of the challenges we've faced with Mailgun as we feel sharing our experience may be beneficial to others.

Firstly though we should explain why we use a third party email service to begin with instead of self-hosting and delivering our own emails directly. Put simply it's to guarantee our emails actually get delivered. That isn't to say we couldn't send our own emails from our own server infrastructure as we certainly could but with the way so many email services are setup the deliverability relies on them trusting your reputation as an email sender.

There are many factors that go into an email servers reputation and one of the biggest factors is the IP Address it sends emails from. If you're sending email from an IP Address which is shared with spammers your emails are more likely to get bounced or blocked by the recipients mail provider. And some spam-blocking services that these mail providers use don't only list singular addresses but entire subnets, zones and even service providers.

This can create a headache for developers like us when we just want to make sure our legitimate email that you expect to receive from us actually arrives and so services like Mailgun, Sendgrid and Amazon SES are paramount to making sure the emails we need to send actually arrive in your inbox.

So why the move from Mailgun? - Well over the past year we've had many occasions where Mailgun's shared address pool has been blacklisted by major email providers including Google Gmail and Microsoft Outlook. Each time we have had to manually intervene raising tickets at Mailgun to receive a different IP Address. Often their responses have been extremely delayed and without much care to the situation or its importance.

On top of that their business model recently changed and their pricing per 1,000 emails sent is now 10x that of Amazon SES while offering an inferior product. In-fact just to get a dedicated IP Address at Mailgun it now costs $80 USD per month while at Amazon the same is only $25 USD per month.

For us the pricing isn't the issue though, we understand the value of a premium product but we felt that Mailgun failed us on too many occasions and their lack of care and attention to their own business negatively affected our own and caused a number of our customers to email us in dismay that they missed query overage notices, late payment notices or emailed our support questions and did not receive any replies from us.

They made us look bad to our own customers because they couldn't do the one job their entire business is setup to provide. This isn't meant to be a review of the Mailgun service but if you're reading this and considering them we would highly recommend going with one of their competitors.

So from today all email we send will go through the Amazon SES infrastructure and we are confident our deliverability rates will be high once again. In addition to this we would like to sincerely apologise to all our customers who were affected by the email problems we've experienced over the past year and in particular over the past two weeks which have been markedly bad and the reason we've spent the extra engineering time on changing mail providers.

Thanks for reading and have a great day!

Incentivising Security and updated Terms of Service

Image description

Here at proxycheck.io we take account security very seriously which is why we only allow you to login to your account using your email address once you have protected your account with a password and why we offer strong two-factor authentication. It's also why we always email you when any big changes are done to your account such as changing your email address, password or API Key.

We do all this because we know that when you implement our service into your own if someone were to gain access to your proxycheck account they could wreak havoc with your services availability.

And yet when we look at the percentage of customers who have protected their accounts with a password we were surprised to see that only 35.09% have chosen to do so. With that being just over one third we think we can do better and so today we've launched a new feature to the customer dashboard which gives you an extra custom rule for securing your account with a password and another custom rule on top of that if you've gone the extra mile to attach a two-factor authenticator to your account.

Image description

For those of you on our free plans this means you'll go from 3 rules to 5 which is quite an increase for free! - These extra custom rules are accessible to all customers including those who already have custom rule plans, they'll simply be added on top of what you already enjoy.

The second thing we wanted to discuss is our Terms of Service. They have been updated and none of the terms have actually changed but we've expanded on the terms with detailed descriptions which brings the page more in line with the look of our privacy policy and GDPR compliance pages including our change log feature so you can stay informed about changes to the page.

June Newsletter

Image description

Today we sent out the first newsletter of this year to users who have the "New features and improvements" email toggle enabled within their dashboard. This has again been the widest distributed newsletter so far with 50% more customers toggling the option on within their Dashboards than for our previous November 2019 newsletter.

If you didn't receive the newsletter but would like to read it you can do so here on our website.

We've made quite a few changes since November 2019 when we sent our last newsletter. We only publish two per year so you can expect our next one around December this year.

Thanks for reading and have a great week!

New Export Log Options, New Partner Relationships & Other changes!

Image description

Late last month we remade the download feature built into the positive detection log within customer dashboards. Previously when clicking the download log button you would receive a basic text file containing your most recent log entries but we received feedback from customers that this text based log was limited in its usefulness and more format options were desired.

We completely agreed and so now when clicking the download button a new menu will open which lets you export your log entries as text, json, csv and html as seen below.

Image description

We've also added a time scale selection dropdown which lets you limit how far back in time the log goes which we know will be a welcomed addition especially for users with very large accounts that incur a lot of positive detections.

Image description

The second thing we wanted to talk about were some new data brokerage deals we have made with other entities. Over the past year we've courted many agreements which have granted us access to attack information that most companies in our space don't get access to. This can range from single occupant firewall logs all the way up to attack logs from entire datacenters.

Last month we began utilising data provided from several new relationships and the affect on our dataset is already being felt through increased detection rates of both proxies and compromised servers. And in-fact we've seen our detection of emerging threats increase substantially just due to having access to so much more data where we can observe both individual addresses and entire ISP's performing attacks across the wider web.

Image description

In addition to the above we also wanted to detail some behind the scenes fixes and quality of life improvements we've made over the past couple of months across both our API and website.

  • Our custom syncing system has been beefed up substantially resulting in faster and more reliable syncing between nodes.
  • We corrected an encoding issue on the v2 API endpoint for City names
  • We corrected a decoding issue on the Threat pages for Region names
  • We corrected a whole bunch of minor visual bugs around the dashboard
  • Various plugin documentation was updated with new plugins, screenshots and descriptions
  • We're now self-hosting everything on our website (fonts, js libraries, icons etc) for reliability, performance and privacy.

And that's all we wanted to fill you in on today, thanks for reading and we hope everyone is staying safe and healthy!

Introducing The Detection of CloudFlare Warp

Image description

On April 1st 2019 CloudFlare announced that they would be offering a VPN service called Warp but instead of focusing on customer privacy by hiding users IP Addresses they would instead focus on speed by utilising CloudFlare's servers and security by encrypting all traffic exiting the users device on its way to CloudFlare's servers.

This differs from traditional VPN services in that they usually focus their marketing on explaining how they hide your IP Address to provide privacy. So last year when CloudFlare announced the Warp beta for iOS and Android users we did not at that time detect CloudFlare as a VPN provider because every website visited by users of Warp had their original and legitimate IP Address sent to the website they're viewing in a custom HTTP header by CloudFlare.

Essentially the service didn't provide anonymity so we felt detecting Warp wasn't justified. However over the past year things have changed somewhat. Warp has opened up to more users on more devices, Windows and Mac betas are being conducted and the service can now be used to access more than just websites.

Which means if a user uses Warp to access other kinds of services (FTP, SSH, RDP, VNC, IRC, Email, Game Servers etc) then those HTTP headers containing the Warp users real IP Addresses are not sent by CloudFlare making those users essentially anonymous.

And so it is for this reason that today we've flipped the switch and are now detecting CloudFlare Warp as a VPN service and our API will now detect and display these IP Addresses as VPN's.

But as CloudFlare is a CDN (Content Delivery Network) and many websites use CloudFlare you're probably wondering if you use proxycheck.io and CloudFlare together will you be negatively impacted by this change in detection.

The answer to that is no. You shouldn't see any difference and Warp users will still be able to access your website like normal if you're using CloudFlare because if you've implemented CloudFlare properly it will present to your website the real users IP Addresses and you would only ever send those to proxycheck.io, you would not send CloudFlare server addresses to us to be checked.

And secondly we've implemented a system whereby we only detect Warp and not CloudFlare's CDN reverse proxy addresses so even if you've misconfigured your implementation of CloudFlare or our service you will not be negatively impacted.

We know that you've all wanted this change to be made because you've told us via email for some time. We've been constantly keeping an eye on the situation with Warp since last year and we've decided now is the right time to enable this detection for all customers. And of course if you want you can still override this detection in your dashboard with the Whitelist feature or a custom rule but as we say there shouldn't be any negative consequences of this new detection being enabled.

Thanks for reading and we hope everyone is having a great week!