Our 2022 Retrospective

Image description

At the end of each year, we like to look back and discuss some of the significant changes that happened to our service and this year has seen some of the biggest changes since we started. So without further adieu let's go through the major milestones.

Back on January 2nd we launched managed rules which are like our previously available feature called custom rules but differ in that we (proxycheck.io) manage the conditions of the rule for you.

This feature enabled you to add rules that have changing data (for instance rules that target a specific organisation) and have us update the rule over time for you.

Image description

Next on January 19th we introduced easy plan alterations. This was a major change in the flexibility of our service enabling you to upgrade or downgrade your plan at any time and receive credits or deductions when changing plans, always to save you money.

Image description

On January 21st we added a major improvement to custom rules called continued execution which allowed your rules to work more like filters allowing multiple rules to work on a single query instead of stopping after a single rule was triggered.

Image description

All of the above concluded a very busy January but February was to bring the biggest change to our infrastructure since we first started. Namely our European Cluster Refresh on February 10th.

Image description

This was as the graph above illustrates a ginormous performance increase for our infrastructure delivering a 4.42x performance improvement when comparing our four new servers to our six previous ones. This performance increase was so great that we haven't needed to add more servers since February 2022 when our new infrastructure was deployed.

Image description

That upgrade not only delivered astounding CPU performance but also quadrupled our system memory and doubled our storage quantity. And speaking of storage we eliminated the last hard disk drives from our infrastructure with this upgrade, switching them for the fastest enterprise grade U.2 NVMe drives on the market and in doing so dramatically reduced our storage latency and increased our storage throughput.

Our next major feature was disposable email detection which launched 5 days later on February 15th. This was a major departure from our core service of detecting anonymous IP addresses but we felt it was close enough in line with what our service stood for to offer it.

And put simply, disposable emails were a nuisance that even we were dealing with daily and so we saw a great synergy between what we already offered and what detecting disposable addresses would accomplish. It has been a great feature for us that has opened our service up to new users.

On April 20th we introduced the ability for customers to pay invoices themselves manually. This was mostly brought about due to new Indian banking regulations that disallowed automatic subscription payments to international entities. We spruced up our invoicing UI during this update as well.

On May 25th we released a major update to our whitelist/blacklist feature that had been present since the Dashboard was first added and had seen very little love since that time. In fact, the original whitelist/blacklist feature was so early in our deployment that it was sticking out like a sore thumb and because of that we wanted to redo it.

Image description

One of the things we thought about during the redesign process was how can we make this feature even more useful while maintaining its class-leading ease of use. One of the things we did with the original listing feature is we gave you a large canvas in which you could put anything you wanted, essentially a big text document where you could write anything without needing to fiddle with UI controls or place things into pre-defined categories.

Our competitors often operate a whitelist/blacklist feature as a series of input fields where you enter one address per field requiring you to keep clicking to add new entries often with a small limit to how many entries you can have in total.

We never did it that way because we felt that was a very poor user experience. It requires far too much input from the user when what they want to do is just copy and paste a large list of addresses in one go.

So when it came time to refresh the interface we knew we wanted to maintain that ease of use however simply having a large text field as we had previously didn't look attractive. So what we did was containerise the lists into what we now call "Custom Lists".

Image description

Above is an expanded view of one such list with all the various controls. This new view enabled easier exporting, re-arranging, a nicer visual appearance, list naming and even creating lists that wouldn't be used for Whitelisting or Blacklisting at all but instead would only be utilised by Custom Rules, an earlier feature we introduced previously.

One other major thing it introduced was downloadable lists. We had customers tell us they liked using the Blacklist feature for example but they found it labour-intensive to keep it updated. Although we did add an API to manipulate White/Blacklists many years ago that does require coding experience.

Image description

And so as the above image illustrates we added a new Automated Mode button which when pressed allows the user to specify a website URL and our service will grab the content of the provided page and use it in their Custom List on a specified schedule. This has been a very well-received feature which is used extensively by our users since its introduction.

On June 9th we reached a Blog milestone with our 200th blog post. We made a post all about dark patterns in design (deceitful user-interfaces) and how we shy away from those practices. It made for an interesting topic.

On August 12th we launched a new site-wide changelog page allowing for easier tracking of new features, improvements and fixes across our entire website.

On September 2nd we added Time Zones to the API result a much-requested feature that lets you see an IP's local timezone.

On September 12th we finally added support for paying for service with a myriad of cryptocurrencies. A much-requested feature since the beginning of our service, this has been a net good for the service, we've processed many payments using Bitcoin, Litecoin, Ethereum, Tether, USD Coin and Dogecoin since its introduction.

Image description

Above are just a few of the crypto coins we now accept for service.

Then as if the month wasn't busy enough we launched a better way to handle your API key changing on September 17th. This was a requested feature for enterprises that like to rotate their API keys regularly but didn't want to have a second of downtime while they did it.

Image description

On October 17th we introduced Postcodes to the API. And much like the timezone feature mentioned above, this was a user-requested feature that we spent a lot of time bringing to the API, it was important for us that it was very accurate before we launched.

And that brings us to today. There are many things we launched this year that were not user-facing. For example, the backend system that we use to shuffle data between servers was completely rewritten. We managed to significantly reduce CPU usage dedicated to database synchronisation through this effort.

We also overhauled how we scan for new proxies and traverse VPN companies' infrastructure. Both for speed and reliability reasons. We also worked heavily with our partners to improve our methods of extracting data from their infrastructure whether they provide it to us directly through our import system or whether we go to them and download the data from their servers.

Another thing we focused heavily on this year was false positives. As mobile internet access and CG-NAT becomes ever more prevalent we have to do more in discovering shared addresses and scrubbing them from our database, we made huge strides in this area and as a result reduced our false positive fate significantly.

As another year comes to a close we got a lot of things done. The physical infrastructure upgrade, custom lists and the accepting Cryptocoin are personal highlights.

We hope everyone had a wonderful festive holiday and we look forward to bringing you even more great stuff in 2023!


Preparing for PHP v8.2

Image description

A few days ago the PHP foundation announced that PHP v8.2-stable will become available on December 8th. This is a major update to PHP which extends the language with new features while improving both performance and resource usage.

Some PHP History

Like most PHP releases this is important not just for us but for the wider web developer community as PHP still drives 77.5% of all websites as of November 2022 (source: w3techs.com). To put it simply, PHP continues to be a pretty big deal.

If you've been reading our blog for a while it should come as no surprise that we make extensive use of PHP for our website and API. This was a bet we made way back in 2016 that PHP as a language would continue to be available and actively maintained.

We looked at other projects at that time built on PHP such as WordPress and what we saw from other developers gave us confidence to move ahead with it. Now we're many years away from that decision and we don't at all regret it, the language has been great for us and continues to improve in every way.

The PHP Foundation

Just a quick sidebar, the PHP foundation just celebrated its first year in operation, you may be thinking that's a typo as PHP has been around since 1997 but it's a fact. Last year in November 2021 the PHP foundation was created and will sponsor the design and development of PHP going forward. I would say their first year has been a great success.

Our current PHP deployment & preparations

So with the history out of the way let's focus on PHP v8.2. Firstly we should mention that we like to stay current, we're using PHP v8.1.13 which was released only a few days ago and that's because every new PHP version brings with it performance improvements, code syntax updates, new features and lower resource usage. All of these things are great for our API as it means we get them for practically free on a regular basis.

Now I say practically because sometimes there are changes which break some prior code. The PHP developers sometimes need to break some functionality to move forward and this is where our preparation for PHP v8.2 comes into play.

Like when we updated to PHP v8.1 that we mentioned in this blog post almost exactly a year ago we need to test and validate that our existing code still runs and runs well. We've been doing that using the PHP v8.2 release candidates. So far it looks good and we believe we'll be able to update to PHP v8.2 quite quickly in December.

PHP v8.2's performance improvements

As we mentioned above one of the things new versions of PHP regularly provide is performance improvements and PHP v8.2 is no different. According to phoronix.com there is a healthy 2.5% average performance improvement over PHP v8.1. And we should note this test was performed very early in the v8.2 development cycle, in-fact it was conducted back in May 2022.

In our testing with the latest release candidate we've seen a better average for the specific functions we make use of in our code and when extrapolated across the billions of requests we answer those performance improvements matter. Essentially the faster our code can execute the more requests we can handle per second on each of our servers.

PHP v8.2's new features

The last thing we wanted to talk about was the PHP v8.2 features we're excited about.

Firstly, we're gaining readonly classes. Pretty self-explanatory, we'll likely make use of this for some of our code data fills that we want to be immutable. Secondly, we're getting a new randomizer class which is essentially a better random number generator with swappable engines. Thirdly, redactable parameters in backtraces. This allows for sensitive data to be removed from error reporting before it leaves the PHP runtime, improving privacy and security.

There are some other changes we like but they mainly deal with alterations to preexisting functions and are a bit too nuanced to explain briefly here.

So that's our PHP update. As we mentioned at the start PHP v8.2 launches on December 8th and we fully intend to be upgraded next month. There are some great performance wins in this release and PHP as a language is looking very healthy, we're looking forward to using all the new features in the coming year and seeing what PHP 9.0 brings!

Thanks for reading and have a wonderful week!


Celebrating Account Deletions

Image description

No, the title of this post isn't clickbait. We are indeed celebrating account deletions. It was one year ago on the 7th of October 2021 when we released our account deletion feature that allowed our customers to completely erase their accounts and all associated data from our service.

And it's also just over a year since we added automatic account deleting which is where we schedule the deletion of unused accounts after a year of inactivity (with a notification email sent to the account owner of course).

Since then we've had 38 users choose to erase their accounts themselves and we've conducted 8,083 automated deletions for inactive accounts (created between 2017-2022). The total number is staggering and shows the scale of the privacy burden we all face when we signup for services. There are entities out there that will never delete your account for you even when it's clear you're no longer using their service and we think that's just wrong.

When it comes to our automated deletions, we send our customers an email after a year of no activity to let them know we'll be erasing their account but if they need our services in the future they can just signup again, even with the same email address as we don't have it stored anymore once we erase their account.

So why aren't more companies performing automated deletions of abandoned accounts? Well.. as you are probably assuming, customer data is a very lucrative "product" and many companies are looking to increase their revenue by either remarketing to prior customers (sending you emails with advertisements for products and services) or selling your personal information to a broker who will then bundle your information up with many other individuals to be sold to yet more entities.

We don't think that's morally right, which is why our privacy policy is so clear about where your data is stored and who has access to it. We're not dealing in mile-long policies that only lawyers can understand. And it's also why we take an active role in minimising how much of your data is available to us, when it's clear you aren't using our services we delete your data, it's as simple as that.

If you would like to learn more about how we're treating our customers like human beings please take a look at our recent 200th blogpost which was all about dark patterns in software design and how we shy away from those deceitful practices.

Thanks for reading and have a wonderful weekend.


Introducing Postcodes!

Image description

Today we've updated our API (May 2022 branch) to support Postcodes. This has been an often requested feature going back multiple years and we're excited to finally launch it for everyone today.

Like all of our other location data accuracy is of the utmost importance to us and a difficult problem as the resolution of a location increases. We spent more than a year testing and verifying that the postcode data we'll be providing through our API is as accurate as our preexisting region and city data.

We also needed to be mindful of the security implications inherent in offering postcode data. Postcodes cover a much smaller physical area than city or town names and that invites privacy and security implications for those having their IP addresses checked.

To ease that concern we'll be providing more resolution than city and town names but less resolution than street names, essentially our postcodes will be for the general area only and not for individual streets which we feel is a happy medium.

So that's the update for today, we know some of you really wanted this feature and will no doubt act fast to integrate it into your sites and services. We look forward to contacting many of you today to let you know that postcodes are now available.

Thanks for reading and have a wonderful week.


High Availability Improvements

Image description

Today we're introducing some availability improvements that assist you in keeping your services fully available to your customers while utilising our API. The first of these changes is to do with API key management.

We've heard from some customers that they like to rotate their API keys on an annual basis. This is a good idea for security as it guarantees any leaked keys are no longer usable. But the problem has been if you change keys there will be a time frame where your software is still using your old key before you can input the new key.

This can result in denied queries or queries which don't utilise your custom rules or custom lists. For this reason, we've had a feature request which allows the old key to still be used after a new key is issued. And that is exactly what we've made available today. From now when changing your API key you'll be presented with the interface below which lets you specify how long your old key will remain usable.

Image description

You can still choose to revoke your previous key immediately upon generating a new key but in addition, you can now choose to keep your previous key active for between 5 minutes and 8 hours by selecting a time frame that fits you from the dropdown box.

In addition to this change, we had another feature request to improve our Dashboard usage API so that it indicates if a burst token is currently in use or not. We've added this too as shown in the example below you'll now see a binary 0 or 1 to indicate if a burst token is active.

{
    "Burst Tokens Available": 6,
    "Burst Token Allowance": 6,
    "Burst Token Active": 0,
    "Queries Today": 1641,
    "Daily Limit": 640000,
    "Queries Total": 1588717,
    "Plan Tier": "Paid"
}

This brings the same activity status for your burst tokens to the Dashboard usage API as you can view in the Dashboard itself like in the screenshot below.

Image description

So that's all the updates we have for you today. We just wanted to reiterate that both of these feature changes are the result of direct feedback from our customers. If you have an idea please don't hesitate to contact us as we may just add it for you!

Thanks for reading and have a wonderful weekend.


Introducing a new way to pay

Image description

Today we're introducing a highly requested way to pay for a yearly plan, cryptocurrency. And we're not just accepting a single type of coin but over fifty of the most popular crypto coins available. Below are just twelve of the most common ones we support.

Image description

You may be asking why are we now (finally!) offering this way to pay. Put simply we've spent a lot of time thinking it through. Accepting cryptocurrency isn't as straightforward as the other payment options available and we wanted to take our time so we fully understood the nuances of receiving and holding cryptocurrency.

We know it has taken us a while to get here but we think all the people who wrote to us over the past several years about accepting cryptocurrency will be happy with the way we've decided to move forward with it.

And that is because we're not just accepting one popular coin, we didn't want to pay lip service to the idea of accepting crypto. And so we're supporting a large selection of different coins. The ones we noted above are just the most significant coins by market cap and so represent the most common coins people are holding but we will be supporting a vast variety of coins outside of these, simply ask us if the coin you want to use is accepted.

And like with our Bank Card, Apple Pay, Google Pay and PayPal payment options you're afforded the same level of service when paying with cryptocurrency which means you can upgrade or downgrade your plan mid-way through a plans lifecycle and you're entitled to the same 30-day full refund policy.

If you would like to pay via cryptocurrency please check the Paid Options tab of your dashboard here, where you can find all the details, we will usually fulfil plans purchased via PayPal or Crypto within an hour during business hours.

Thanks for reading and have a wonderful week!


Time Zones added to the API output and Custom Rules

Image description

Today we're announcing a new piece of data we're exposing through our API called time zones, this is as the name suggests a way for you to view the time zone of an IP address you're checking with our API.

We've added time zones because we see it providing a lot of benefits to our customers who want to tailor services to users within specific geographic areas without targeting their country or region.

Knowing a user's local time zone can be hugely beneficial to businesses, just like in our own everyday lives where we plan our day around specific times, so too do businesses. Until now customers have been reliant on the user's browser to show local times but this isn't always an available metric. Our API is not only used on websites where the browser can provide that information.

In addition to exposing the time zones in our API, we've also added it to the Custom Rule feature which means you can create conditions and condition groups which utilise time zones for targeting. We've chosen to follow the iana.org time zone format which means you can easily parse the time zones we provide using standard and common libraries within all major programming languages.

We've updated our API documentation including the test console to showcase the new data and we hope you'll experiment with it!

This feature began as a request from one of our customers which we felt had broad appeal and great synergy with the other data we already make available. If you have a great idea please get in touch via our contact page, we love to hear from you.

Thank you for reading and have a wonderful week!


New unified changelog interface and other news

Today we've added a new changelog page to the website which allows you to quickly view the changelogs from any of our pages in a single unified interface. The main benefit to this is you no longer need to visit individual pages to read about changes and it's easier to read them in a full screen layout as illustrated below.

Image description

Like many of our features this has been on our roadmap for a while, we are very proud of our transparency when it comes to feature changes, additions and fixes. We believe strongly in documenting when things change especially when it comes to our pricing page, privacy policy, terms of service and GDPR compliance which are pages you don't traditionally expect to find changelogs.

We hope you'll enjoy the new page, it does look very colourful and it may even be interesting to read how some features like the Dashboard have evolved since 2018 when we first launched it and began logging the various upgrades and changes made to it over the years.

Looking past this new feature we did want to discuss some other things. Since June we have been performing a lot of backend changes to keep the service strong for the future. Mostly based around database throughput and synchronisation within our cluster of systems.

We've also been improving the sites security against client side attacks that could be leveraged against our customers, specifically XSS attacks. Forms, buttons and other basic functionality has been hardened to make it more difficult for potential attackers to hijack your login session with our dashboard through malicious links.

In addition to that we've for the first time implemented a Dashboard API request limit. Dependant on what you're accessing and how frequently you're accessing it you may be subjected to a 1 to 2 request per second limit for a brief time period.

Just to reiterate, this is for the Dashboard API where you manipulate your account settings programmably, view statistics and download logs. It does not apply to the proxycheck v2 API that you use to check addresses with which has its own per-second request limits published within our API documentation page.

We've added these Dashboard API request limits to help secure our infrastructure against the possibility of resource depletion attacks. We don't believe these introduced limits will impact any of our legitimate customers.

The final thing we wanted to discuss was inflation, the cost of living increases and the prices of energy. We like most businesses are subject to market forces and that does mean our costs are increasing, we're seeing significant price increases in server hosting due to European energy prices but we want to reassure all our customers, you need not worry as we do not raise prices on currently subscribed customers, period.

In addition to that we're not planning to increase the prices of our plans for new customers at this time, we're instead going to absorb the increase in operating costs so don't feel pressured to signup or upgrade your paid plan as no price increases will occur for the remainder of this year.

We should emphasise that traditionally we've performed pricing changes in January and that's still the case. And again any pricing changes would only apply to newly started plans or changes from one plan to another. Those already subscribed to a plan would not be subject to any price increases.

We know we've been quiet all throughout July without a blog post but rest assured we are working on things behind the scenes. We've been bringing on more data partners, increasing the robustness of our backend software and improving security.

Thanks for reading and have a wonderful weekend!


Regarding Downtime Today

This is just a quick post as the situation is still developing.

Around 07:30 AM GMT the CloudFlare network began to fault worldwide and our site aswell as all others on the CloudFlare CDN became inaccessible to the internet. We apologise for this and unfortuantely we were unable to mitigate the problem due to our heavy reliance on CloudFlare for DDoS protection and Load Balancing.

Around 08:12 AM GMT service began to be restored and CloudFlare is continuing to update their status page which you can view here: https://www.cloudflarestatus.com/incidents/xvs51y9qs9dj

When CloudFlare shares their post-mortem we will update this post with a link to that. Once again we deeply apologise for the unexpected downtime, it's not something we take lightly and we appreciate your patience and understanding.

UPDATE 1:

Now that we're able to see full analytics for the time period in question we can see that only around 50% of our traffic was affected by the CloudFlare downtime, the other half continued to be answered normally. This appears to be due to the CloudFlare issues only affecting specific regions and points of presense for their network. We are continuing to evaluate things but as of this update everything is fully operational.

UPDATE 2:

CloudFlare has now published their post-mortem report: https://blog.cloudflare.com/cloudflare-outage-on-june-21-2022/


Dark Patterns: A topic for our 200th blog post

Image description

Today we wanted to do a special entry for our 200th blog post and it's all about dark patterns in user interfaces, what they are, how you can spot them and how we reject their use in our product.

So first lets describe what dark patterns are, in short it's a way to describe a deceptive user interface which has been designed in a way to trick users into doing things they don't want to do or to make it unreasonably difficult for them to access or perform actions that they want to do.

Some examples of this would be an e-commerce website advertising a sale on an item when it hasn't been discounted from its usual price or removing a discount once an item is added to your shopping cart. Another example would be advertising free shipping but then when you checkout there is in-fact shipping or another service charge that would equal the shipping cost.

A common example of dark pattern use in paid subscriptions are when a service makes it easy to signup and pay but difficult to cancel a plan or delete your account.

There are countless examples of dark patterns all over the web and thankfully the law isn't blind to these practices and some states like California in the USA have passed stronger consumer protection laws which encompass deceptive user interface design.

But there is always more that can be done. Here at proxycheck we have rejected dark patterns since the start and we would like to list a few of the ways we've accomplished that.


When you signup we only ask for an email and we do not sell or provide your address to any third parties besides our mail carrier. We don't abuse your email with spam and by default our marketing emails are opt-in instead of opt-out, meaning they're off by default.

When you don't need to use the service anymore you can quickly and easily access an account deletion button within your dashboard. It's always fully visible in the top right corner of the settings tab which is the default tab.

You can export all of your data at any time through the dashboard, no need to contact any customer service people to get a copy of your data.

When you want to upgrade or downgrade your plan you can do it yourself from within the dashboard and all changes are prorated to save you money.

When you want to cancel a plan you can do that from within the dashboard with two clicks from the paid options tab.

Clear pricing that we stick to, you are never charged more than you agreed to and we don't increase prices for subscribed customers even if the plan they're paying for increases in price while subscribed.

Clear and descriptive change-logs available at the bottom right of every page which detail all the changes made to that page, nothing deceptive or hidden here (also we encourage archive.org to scrape our pages to provide an independent log of our changes).

Easy access to refunds even mid way through a subscriptions life cycle with 30-day full refunds since your latest payment, not your account signup date.

Email alerts a week before we're going to bill you which are turned on by default and cannot be disabled for yearly subscribers so they won't get caught paying for something they're no longer using. (And remember you can request a refund if this happens to you)


For us offering these features was not about following a law, we've done most of this before the term dark patterns was even coined or legislated against. For us this is about delivering fairness. We do not look at our customers as an exploitable resource, we see them as valued partners.

This is why we've continued to deliver innovative new features like CORS support, Custom Rules, Burst Tokens, Custom Lists and more at no extra cost. It's why we've expanded the amount of data we provide and why we continue to add features customers need like upgrading and downgrading their plans, deleting their accounts when they no longer need the service and paying their bills outside of a recurring subscription.

At the start we mentioned this is our 200th blog post and that's true. We started this blog on June 20th 2017 and since then we've written about a huge number of topics, service changes and feature additions, we look forward to doubling that number in the coming years and we hope you'll be along for that ride.

Thanks for reading and have a wonderful week.


Back