Dashboard Speedup

Today we've been spending some time going through and auditing code all over the site. Mainly CSS related but we've also been looking at ways to improve page load times by reducing image sizes, moving scripts around that cause slower page painting and so forth. One of the areas we looked at was the Dashboard. Sometimes on initial loading it can take some time before it begins to show any content beyond the navigational bar and this was due to our use of Javascript to load the initial content you see when it first loads as-well as handling the switching of separate tabs within the Dashboard.

So today we've gone through and altered how this code works so the content you initially see when you load the Dashboard (or subscribe to a paid plan where it shows you the Paid Plans tab by default) now load instantly.

This has a dramatic effect on how the page feels, it loads practically instantaneously now. No one has actually complained or requested this change but it's something we noticed while auditing the website as we do from time to time.

One of the best parts of our service currently is the customer dashboard, it is our second most visited page after the home page and so it's important to us that it's well built. We feel the code behind the scenes is written incredibly well and will be easy to maintain going forward so we're not planning a rewrite of the Dashboard any time soon, just maintenance and new features like our recent ASN support and improved Stats exporting.

Thanks for reading and have a great day!


New Code Examples

Today we've expanded our code examples page with new code snippets for Node.js, Python and Ruby. We've also added a curl command line example. These are all very simple examples to help get developers started and we are including your API key in all the examples (including the C# one now) to get you up and running faster. All of these examples were submitted to us by users of our API and we're very grateful for your contributions. If you have other languages you'd like included or you would like to expand upon any of the examples we've provided please submit those to us through our support email address [email protected] we are more than happy to feature your work on our examples page for other users to benefit from.

Thanks for reading and have a great day!


New Service Status Page

When we first added the Status Page it was a great resource to quickly view if any of our server nodes were offline and their load conditions. But as our service has evolved we've added so many new things that the status page wasn't serving our needs fully.

So today we've introduced a brand new service page written from the ground up just for us. Like all of our services it runs on our cluster so the chances of it being unable to load are very low.

As you can see it has a new interface that lists each of our features separately. And if you hover your mouse over the tables you'll receive a bit more detail about what the service is or what's for.

We are now able to list our backend systems so we ourselves can get a quick look into issues. And our new status page doesn't just display whether a service is functioning or not, it also can display intermittent issues, high load conditions or anything that would disrupt service.

And finally, we can list our individual Honeypots. Allowing us to see how our IP vacuum cleaners are doing. We hope you like these changes, we feel they were necessary and it does give you a more complete picture of our infrastructure. It also allows us to display more servers in less space which will come in handy next year.

Thanks.


Should I be using HTTPS to query the proxycheck.io API?

It's a question we get quite often, what is the benefit of using transport security for API queries. We've offered it since the day we launched but it's not completely obvious why you'd need it so we're going to explain it.

Image description

Firstly HTTPS means Hyper Text Transfer Protocol Secure. When in use an encryption algorithm is used to secure your connection to the server you're communicating with. In our case that is your application server with our application server. All the information our two servers transmit and receive while communicating is now cryptographically secure meaning third parties cannot determine what you're sending us and vice versa.

So why would you need this advanced security for what seems on the surface quite basic API calls?

Well the main reason is, it stops your visitors from being tracked by third parties. In the current political climate we have world powers trying to undermine individual personal security at every opportunity. So when you send the IP Address used by your visitors to proxycheck.io there is potential for a government agency or other organisation to record that interaction, that is if you've not used our HTTPS API endpoint.

We could call this kind of information collection metadata because although they don't know what the user was doing on your website they know they visited it and they can link that visit with their IP Address within a larger database to track that user and build an overall profile of who that person is and what they do online.

That's we feel the main reason you would want to use the HTTPS endpoint. The second reason is for your own account security. If you're making an API request to our service as a signed up customer you have to supply your API key with every request. If your communications with our server are being intercepted it is possible for a third party to grab your API key and begin to make queries against your accounts query allowance.

And that could cost you money or exhaust queries you've already paid for. The only real drawback to using the HTTPS endpoint is the added time it takes to setup the encrypted connection. There are more handshakes and third parties have to be consulted about the accuracy of our encryption certificate which increases the time it takes for your query to be answered.

It is for this reason we offer both HTTP and HTTPS endpoints for our API. We're giving developers the choice. We hope this post has been helpful in explaining why we offer HTTPS to all our customers and have done so from the very beginning. Privacy can only be maintained when we all do our part to strengthen it.


Payment Processing Fixed

Some time between the 17th and 20th of October our payment system stopped loading within the Dashboard. This was caused by a compatibility issue between our CDN Partners Javascript compression technology and our payment widget.

We managed to fix this problem this morning and we are once again able to take new payments. This bug may have also affected customers trying to cancel a paid plan, if you were trying to do that, you can now do so. This issue did not affect customers who were already subscribed, your payments during this time would have been processed normally.

Thank you to the users who made us aware of this issue this morning, we are very sorry it happened and have taken steps to make sure this doesn't occur again in the future.


Enforcing Terms of Service

An issue that almost all service providers will come up against when they offer any kind of free plan is users finding ways to maximise their use of these free resources well beyond the limits imposed by the service provider. For us this manifests itself in two ways.

1. Users signing up for a registered account multiple times under different email addresses

2. Unregistered users performing queries from a large pool of IP Addresses and/or Proxy Servers/VPN's

For the most part we don't mind if a registered user has two accounts. Perhaps you need one for production and one for your development environment. We do not police users that have 2 accounts, especially if you're under the 1,000 query limits on both accounts.

However we have found recently multiple users signing up for 10 to 15 accounts and load balancing their queries. This essentially means they have a 10,000 to 15,000 daily query limit for free when it should only be 1,000.

Another problem is unregistered users performing queries from Proxy Servers. So instead of making 100 queries per day from a single IP Address they are making thousands of queries per day across hundreds of IP Addresses. In-fact we've seen some users utilising more than 1,000 proxy servers in a single day to load balance their queries.

The thing to keep in mind if you're a user that does this is our API is not an unlimited resource so we cannot let the abuse of our API and the disregarding of our Terms of Service continue.

To this end over the past week we have been contacting registered users that have more than two accounts to let them know we've disabled all but one of their accounts. In these situations we always leave the account that has performed the most queries active while the rest are disabled.

We're also tackling the unregistered user abuse issue by now checking that the IP you're using to contact the API isn't a proxy server. If it is, the query will go unanswered. To be clear, this only affects unregistered users, we do not perform this check if you have registered for an account and are using your API Key to make your queries. We're also not blocking VPN services from making unregistered queries.

So if you wish to contact the API through a proxy server you can still do so, you'll just need to signup for an account and supply your API Key with your queries.

We hope that these changes will not disappoint too many of you. If you're a registered customer and adhering to our Terms of Service you will not notice any change to your service. If you're an unregistered user you may find your queries take a few milliseconds longer while we verify you're not accessing our API from a Proxy Server.

Thanks for reading and have a great day. If you have any questions please feel free to contact us.


ASN support added to Whitelist/Blacklist feature

Recently we've had customers ask us if our Whitelist/Blacklist feature supports ASN's and if not, when will support be added. Today we've added full support for ASN's to both the Whitelist and Blacklist feature. We've also enhanced the API responses when you perform queries which are affected by either of your custom lists.

So firstly the way you add ASN numbers to be White or Blacklisted is a new ASN format like this: AS1928 So for example here is a screenshot showing some examples:

So to place ASN's you simply follow the format which is AS###### you can place as many ASN's, IP's and Ranges in the White and Blacklists. All will be lifted from the box and you can still place comments besides your entries.

Now we've also altered the JSON result from the API when one of your Whitelists or Blacklists has been utilised for the IP you're checking, this is how that looks:

{
    "node": "HELIOS",
    "ip": "62.205.245.173",
    "proxy": "yes",
    "type": "blacklisted by AS28843",
    "query time": "0.001s"
}

Previously it would only say blacklisted or whitelisted, now it actually will tell you what triggered the type of detection, it can list individual IP Addresses, Ranges or the ASN numbers you've specified.

We hope you enjoy these changes they are a direct result of your feedback.


Brand and Logo Usage

Recently we've had a lot of enquiries about the usage of our name and logo within customer websites and software so we thought it would be a good idea to make a blog post about our stance.

Firstly we welcome you to tell people that you use our service, it's good for us whenever people are exposed to our branding. So to put it simply feel free to place our name or logo on your products.

We do have a few minor stipulations though. Please do not modify our logo beyond resizing it or changing the colour to monochrome to fit within your products design. Don't use our logo for your application or make our logo bigger than your own logo so that there can be no confusion about the connection between us and your product.

Finally don't make it seem like you are us or endorsed by us or that your software has been made by us, you should always make it clear that we're seperate entities.

To assist you in adding our branding to your product we've provided our logo here: https://proxycheck.io/images/proxycheck.io-logo.png it does feature a transparent background and it's 547 x 559 pixels in size. Please rehost the image instead of linking directly to it.

Thanks for reading and have a great day!


Expanding our Code Examples page

Last year when we added our Code Examples page we started off by offering a basic PHP script which you could insert into your webpages. We later expanded it with a proper function and opened a GitHub account.

Since then we've been looking at ways to expand the types of software that work with our API, our open approach to including third party developer clients and third party code on our example page has been a great way for us to increase the exposure of our API and to allow our customers to use the API more easily.

Earlier this year we added several more Minecraft plugins to the Examples page, four in-fact. And those plugins have resulted in many new people using our API.

Two weeks ago we further expanded the examples page by giving it a new interface that allows us to share many more code examples. One of the latest ones we've added is a C# console app assembly example written by a third party developer.

In exchange for writing that example he received a paid tier account for a period of five years with the volume of queries he needs for his software. We love doing deals like this because it benefits both parties. He had already written the integration code for his own software and simply by sharing it, he has secured access to our API for several years.

We want to do more of these kinds of deals and so if you have some code examples for C++, C, JAVA or Node.js please let us know about them, we would love to give you a paid account in exchange for sharing your code, plugin, class or function with us and not only will we feature it on our website but you'll receive full credit and we're more than happy to link to your personal website, GitHub projects and Twitter account.

We've spoken previously about third party software that integrates our API and our stance is still the same, you can integrate the API however you wish in any software you release and you are free to monetise the software you make, you don't owe us a penny of your revenue. We make our money by selling access to our API and so for us the client software is not a profit centre. We can prove our stance by actions as we do advertise a paid Minecraft plugin on our examples page which was written by a member of the Minecraft community and is not affiliated with us in any way.

So if you have the skills to make some great client software that you want to give away for free or sell at a profit, to you we say, hitch your wagon to our star, let us worry about maintaining an always accessible and accurate proxy detection API while you make great software.

Thanks for reading this blog post, if you would like to talk with us about anything discussed here please email us at [email protected] we read every response and you'll always receive a reply from us.


Reducing stagnant data

When you operate a data driven service such as proxycheck.io you will come up against an issue where you need to decide at what point data has become old and irrelevant.

For us that means how long should we consider an IP Address bad before we remove it from our database. In the past we would cache an address for a period of 90 days since we last saw it operating as a proxy or compromised server.

But this presents an issue in that addresses are often repurposed and bad services running on compromised servers get cleaned up constantly. So this means it's not always best practice to hold IP data as long as we have been.

So we're extending the duties of our inference engine to not only discover new IP Addresses acting as proxy servers but also to go through our old data and verify that the IP Addresses there are still bad.

This means we're now holding IP's for a minimum time of 15 days down from 90 days. The inference engine will make assessments every day from the moment we first add an IP to our database and then slowly discard Addresses where it has a 100% confidence rating they're safe again.

We believe this will cut down on false positives allowing more of your legitimate users to access your services without being blocked unduly just because they received a previously abused IP Address. This change has been active for a while on our development platform and after positive and accurate results we've engaged the system on our live data.

Thanks for reading this change and as always have a great day.


Back