Custom Rules Update: Condition Groups

Today we're introducing our second major post-launch update to the custom rules feature found within your customer dashboard and it's quite a substantial update, but before we get to that we would just like to tell you about the customer response to the custom rules feature.

Since we released the custom rules feature on July 17th we have received an enormous amount of feedback from customers. Infact it has become the most discussed feature within the conversations we hold with customers. This is why soon after launch we added more conditions and early in August in our first major update to the feature we added output variables.

Todays update focuses again on rule conditions. Previously every condition you added to a rule all had to be true for that rule to be acted upon. This meant you may have had to create more rules than you wanted simply due to your need to account for every possible situation.

Sometimes you just want to create a rule that only applies to a few different IP Addresses, Countries or Cities and you don't want to create individual rules for each scenario. While from the start we supported the supply of multiple values for a single condition (and this is still supported) it was not a great experience to fill such a small box with so much data and it also limited you to one specific condition type for all the values you supplied.

With this new addition of condition groups you can create as the name suggests a group of conditions where only a single condition within that group needs to be true for the entire group to be satisfied. This allows you to easily create rules which apply to a whole bunch of countries or addresses or any of our condition types allowing very powerful rule targeting.

Below is a screenshot where we're creating a rule that only applies to a specific internet service provider in the United Kingdom. But we only want that rule to apply to three cities that they operate in and not the entire country. As you can see we've cleverly colour coded different conditions, pink conditions are required to be true while conditions wrapped inside a blue background only need a single condition within its group to be true.

Image description

Now this rule will only be activated if the ISP and Country matches the entries in pink and also when any one of the three Cities in the condition group are seen. And as you can see on the left within the condition group there is a dropdown. You don't need all the conditions within the group to be the same type. We could substitute one of the Cities for something else like an ASN number or an address range.

Like previously there are no limits on the amount of conditions you can specify for each rule and this extends to condition groups. You could if you wanted create your own whitelist or blacklist inside the rule feature using a single rule simply by creating a condition group and filling it with the Addresses, ASN's or IP Ranges you wish to whitelist or blacklist.

This update extends the foundation that the custom rules feature is built on. No functionality has been lost, you don't need to re-save your rules and the import/export features still work as you would expect. You can even import rules created with the previous version of the custom rules feature without issue.

This feature update has taken a lot of work but we know it's worth it. Customer response to Custom Rules as noted above has been phenomenal and it's our intention to keep adding new functionality over time with backwards compatible updates like this one so you can retain complete control of how our API interacts with your properties.

Thanks for reading and we hope everyone is having a great week.


Lots of Dashboard improvements

Probably the feature that our customers interact with the most is our customer dashboard. Which is why we spend a lot of development time making it better. Over the past month we've added a lot of small enhancements which we would like to tell you about.

  • Icons and Buttons around the Dashboard were updated, we also added icons to buttons that previously didn't have any.
  • Greatly improved the speed at which your query statistics and positive detection log now appear (Avg 1 Minute).
  • Made the Queries Today and Queries Total show the most accurate and up to date volume of queries you've performed.
  • Added back support for customers to update their card details for a currently active subscription.
  • Improved the appearance of the account information table at the very top of your dashboard and moved the copy API Key button.
  • Added more text labels to paragraph blocks so it's easier to quickly read and summarise what a section is for.
  • Added support for custom rule plans so we can offer our largest customers custom rule plans separate to query plans.
  • Added a new custom rule output modifier to the rules feature which allows you to remove an output from the API by name.

These sorts of changes are always available to be read by hovering your mouse over the "Last Update" feature which is present at the bottom right of all our web pages.

Keeping you informed about the development progress of proxycheck.io is very important to us and while we usually wouldn't make a blog post for each of these changes on their own taken together there's a lot of improvements that warranted this blog post.

One thing we wanted to talk a little more about is the re-adding of support for customers to update their card information for a currently active subscription. When the European SCA (Secure Card Authentication) regulations came in we updated our integration to be compliant and during that time there was no way for us to do what's known as capture a customers card information for later use.

Due to that we had to remove the card updating feature from our website until our payment processor (Stripe) was able to provide such an integration which we could use, they were able to do that just ahead of the September 14th SCA deadline and we moved quickly to add that feature back within a few hours of Stripe launching it.

We're aware that some of you were inconvenienced when this feature was removed which is why we made every effort to add it back as soon as we could. As of now we're fully SCA compliant and as a result of that compliancy we now also accept Apple Pay which we spoke about in a previous blog post.

That's all for now, thanks for reading and have a great weekend.


New Dashboard API: Tag Stat Exporting!

When we added the new tag statistics section to the customer dashboard a week ago we instantly received requests from customers for an export API containing this data. We agreed with you that this feature was important and so today we've added it.

If you just want to see the documentation for this new API you can read that right here within our API documentation page.

This API is a bit different from the others, it has some more advanced features like being able to specify start and end unix timestamps to create a viewable time frame. This allows you to really look at tag use between any time period you specify. But we've also kept some more basic time controls like an easy to use days flag for our users who prefer that.

Below is an example of this new API endpoint showing how it can give you some great insight into your tag usage and the IP's triggering those tags.

{
    "www.example.com\/board\/ucp.php?mode=login": {
        "types": {
            "total": 10845,
            "rule": 468,
            "proxy": 10377
        },
        "addresses": {
            "178.137.81.87": 4263,
            "46.119.114.95": 3615,
            "46.119.115.202": 2967
        }
    }
}

With this new API Endpoint we've also updated our own dashboard implementation to use the same API and that brings with it enhanced performance, especially for users with a lot of tagged queries and we've also added the ability to click on entries to view the associated IP Addresses and a button to copy the addresses to your clipboard.

Thanks for reading and have a great weekend!


Dashboard updates: Rule Import/Export & Tag Statistics

Today we made two changes to the customer dashboard the first enables you to download your custom rules for backup, editing or to take to another IP processing company. We've also enabled the ability to upload your own rules in JSON format which makes it possible to create your own rule generator or easily move your rules from the different accounts you may have with us.

Image description

You'll find two new buttons within the global control interface on the custom rules tab of the dashboard just like in the screenshot above.

The second change we've made is to do with the stats tab where we've added a new section to view tag statistics as shown in the screenshot below.

Image description

This feature has often been requested and we're happy to be able to add it to the dashboard today. It essentially allows you to see how often your tags result in a positive detection with a breakdown showing the types of detections that it triggered.

So that's it for this update, we hope everyone is having a great weekend!


Post-Processing Inference Improvements

Today we've released a significant update to our post-processing inference engine which should greatly increase its detection time of bad addresses. The post-processing inference engine is the part of our service that stores addresses we don't think are proxies in a temporary cache so that more data can be accumulated and more processing power can be used to determine if the addresses are in-fact bad.

The new changes today allow the engine to be a lot more aggressive on when it starts making determinations. Prior to today we would convert IP Addresses into one-way hashes (similar to how we hash passwords) and then only make a determination on whether those hashes are proxies or not after we had gathered 8 hours worth of data.

This meant that if an IP was going on a rampage, such as registering on lots of websites, brute forcing passwords and so forth it's likely we would not detect it as a bad address until its 8th hour in operation. For some IP's you really do need this amount of time to get a full picture of its intent as we do not have a complete overview of the entire internet, we only see a small slice.

But for other addresses it becomes obvious in mere seconds that they are bad. And so that is why we've re-engineered our post-processing engine to be a lot more aggressive in when it begins to make determinations. Now it will constantly re-evaluate them based on the actions it's seeing them perform in near real-time allowing us to go from first sight to determination in seconds instead of hours.

Already we're seeing an uplift in detection rate for the worst addresses attacking our customers and honeypots. As our customer base continues to grow, the speed at which attacks are correlated will increase resulting in even faster determinations. Essentially the more our customers and honeypots receive attacks the faster we can identify those bad addresses causing those attacks and share our determinations through our API.

In addition to these changes we've also updated our machine learning model and the inference engine can now hold hashed addresses for a much longer period of time if it feels that's warranted which allows for more data accumulation. This should increase the overall detection rate over our previous model due to it now being able to gather much more evidence.

Thanks for reading this update and we hope everyone is having a great week.


Introducing Variables to the Custom Rule Custom Outputs

That title sure is a mouthful and I'm sure you're scratching your head right now just trying to decipher it! - Hopefully after reading this post you'll have an understanding of exactly what this new feature is. (The Custom Rule documentation has been updated here if you just want to read that instead).

When we added the Custom Rules feature late last month we listed one of its main features as being able to customise the output of our API. Meaning you can provide some values that will be shown in the API result if your rule conditions are satisfied.

For example if you wanted to add an extra field to the API with some text you could do that. But we didn't at that time provide variables which is where you could include some data from the API in your customised fields. Today that changes with the introduction of variables. And we think we've accomplished this in a really simple to understand way.

Put simply we have a list of 14 variables such as %IP%, %COUNTRY%, %ASN% and %TAG% which you can include in your custom output value boxes when you create rules. When the API sees these variable names it will replace them with the data the variable name is for.

So if you use %COUNTRY% and check an IP for Spain that variable will be swapped out with the word Spain. Pretty simple right? - So why are we doing this. Well it's so you can further customise the output of our API to suit your needs and do some clever stuff.

For instance if you wanted to add content to a custom tag you've provided with a query you can do that by writing your new message and then including the %TAG% variable at the end. That will essentially add your custom information to the start of the tag because while you're overwriting the old tag you're also including a copy of it in the new tag.

So if you had a custom rule called "Block Russia" which always detects Russian IP's as proxies and your tags are all like this: "example.com/myblog/login.php" and you wanted to note that in the tag this IP was detected as a proxy because of a custom rule you could supply a custom value like this: "Activated by Rule: %RULENAME%, %TAG%" which would output: "Activated by Rule: Block Russia, example.com/myblog/login.php".

And that would then appear in your positive detection log within your dashboard on our website allowing you to very simply track when a rule was acted upon that resulted in a positive detection as opposed to an organic result that wasn't caused by your rule.

So you can see the power that variables provide and we know you'll find them useful as we're using the custom rules feature ourselves on our own personal websites, forums and blogs and much of the new functionality such as replacing tag content and adding output variables are a direct result of us using our own product. With that said we're still in full custom rule development so keep checking back for more updates!

Thanks for reading and have a great weekend.


Dashboard update

Today we've released a new version of our customer dashboard which spruces up the interface a little bit and adds a more secure way to pay by card and also a new method of paying entirely that doesn't use your card.

So firstly you may notice we've switched up our icons from the previous thin line design to a new dual-tone design which is a bit more modern. We've also made this change to our feature section within our pricing page. We think these new icons look really fantastic, they were done by the FontAwesome.io team of which we are a happy patron.

Image description

Secondly maybe not so obvious is we've added some more animations to the dashboard. The invoices section for instance now smoothly slides out as does the positive log filter. We also corrected some inconsistencies with the UI with regards to margins, button animations and styling. These are all very subtle changes that you may not notice but the overall quality improvement will be felt.

We've also had customers tell us that after being subscribed for a long time the invoice section was getting a bit cluttered so we're making some changes there, we're now only showing invoices for your current or most recently held subscription and by default the invoices will be hidden instead of exposed. You can of course always ask support for a past invoice, they're not lost and we always email you a link to your invoices when we charge you, those links will always remain functional.

So let's get to the big change, the new payment gateway. Since we started charging money for our API we have been using Stripe as our payment partner. They've been terrific and we really love using their service. Recently they announced a new version of their checkout product which moves the payment interface from the website you're paying on (proxycheck.io for instance) to Stripes own website.

This has a major advantage in that whilst we always protected your payment information by opening a secure frame to Stripe (and thus we never saw or handled your card information) there was always the chance that the payment page on our own website could be replaced with a malicious one that contains a payment window which looks identical to Stripes and users wouldn't be able to tell the difference.

That is no longer an issue with the new checkout page which is completely hosted on Stripes own website. And it's just as easy as before, you pick the plan you want and click the Subscribe button you're then transferred to Stripe, enter your payment information and then you're transferred back to us for your plan to be processed.

The new checkout also has some enhancements that were not possible on the previous checkout. For example, it supports SCA which is a new secure initiative coming in September where all payments originating within the European Union will require the card owner to authenticate using their mobile device. We welcome this higher level of authentication as it will significantly reduce fraud for both card owners and merchants like ourselves.

Image description

But that's not all, with the new checkout we also gain support for new payment methods. As of today we now support Apple Pay which means you can use Safari on your Mac, iPhone or iPad to setup a subscription and in the near future we will also be adding support for Google Pay, Microsoft Pay, iDEAL, Sepa Debit and even more payment methods through the new Stripe hosted checkout page.

So that's all the updates to the Dashboard for today. We still have a lot of things on our roadmap yet to come so keep checking back!


Custom Rules Documentation

This is just a quick update to let you know we've added preliminary Custom Rules documentation to our API Documentation page which you can visit through this link.

We will be updating this documentation over the coming weeks and months as it was our original intention to provide a more interactive way to show you how to use the feature instead of just telling you how it works.

But we've put this up earlier than planned because we have been inundated with questions about the feature and it became clear to us that making any documentation available is better than nothing and we can come back and update the documentation with something more interactive at a later date.

So thats it for this update, we hope everyone is having a great weekend.


Custom Rules update, more conditions!

When we released the new Custom Rules feature on July 17th we asked you all to try it out and give us any feedback you may have. And to our surprise a great deal of you have created rules that remain active as of this post indicating you've found the feature useful.

Whenever we make a big feature like this we always wonder exactly how many customers will make use of it and it's not like you all visit our website every day to find out what we're doing so to have so many of you use the feature and so soon after its unveil has been quite surprising.

And with that surprisingly high usage has come a lot of feedback. In-fact we probably received more feedback since launch about this feature than any other. It has been made clear to us that you all love being able to create an unlimited amount of conditions and outputs within each rule. Making that unlimited was the right way to go and we've received a lot of praise for that.

You've also made it clear you want more conditions which is why today we've introduced Latitude, Longitude and Port Number condition targets. The first two new conditions will allow you to create complex geofenced rules by utilising the Latitude and Longitude data we provide within ranges you specify.

Another piece of feedback we received was that you would like some kind of library with pre-made rules for common situations so that you can browse and save a rule to your dashboard which you can use as-is or edit to fit your specific need. We do intend to make this and in-fact it was brought up during the initial development stages.

The final piece of feedback we received is you would like to see documentation for this feature. Some of the condition values you can provide are not obvious and while we have enhanced the status messages that appear to be more descriptive since the launch we agree with you that a full page piece of documentation in the same style as our API Documentation page is necessary. We will be working on this.

So that's your first update to the rules feature. We're of course still accepting feedback for this and all other features so please get in touch with your ideas, we love to hear them.

Thanks for reading and have a great week!


Introducing Custom Rules

Today is a day I've been looking forward to for a very long time because today I get to share with you all our new custom rule system. This is a new dashboard feature that puts you in complete control of how the API responds to your queries with an unparalleled level of customisation.

This feature has been on our long term roadmap for as long as I can remember. It required lots of planning, experimenting and testing, we wrote the feature several different times from scratch until we nailed it.

We know it's going to be a big deal for our customers because you've been telling us so every month for the past two years. You may not have specifically said the words "I want custom rules!" but you definitely said "Can the API respond this way if x and y happens?". Every month we're asked to add more flags to the API that allow specific use cases.

The two most common requests we receive are. Can you add an easy way to do Country blocking on the API side and what if I only want to block the really bad VPN's and not okay ones? - Well with the new custom rules feature you can accomplish both of these things and a whole lot more.

And that's because not only can you alter the logic of the API but you can overwrite responses with your own ones or create entirely new custom fields containing the strings you specify.

So that's enough talking, let's show it to you. Below is an animation showing the global controls for your rules. The extra control buttons you need only appear when you have rules created and they disappear when you don't. We feel this makes the interface more approachable, less intimidating.

Image description

When clicking on the add a rule button a new rule will be added to the page. Below we've added a rule and expanded it to reveal all the configurable options. We've also populated it with three conditions and a single output modifier. Each rule you create can have as many conditions and output modifiers as you like. And we also allow you to specify multiple values for a single condition, more on that in a bit.

Image description

In the screenshot above hopefully you can identify what this rule will accomplish just by looking at it. But essentially if you send an IP to be tested and it's a VPN but its risk score is below 67 then with this rule enabled the API will output Proxy: No. Normally in this situation it would output Proxy: Yes.

Which means this rule has enabled you to only block VPN's which we consider dangerous by utilising our risk score. Now you could create logic like this in your own client software that talks to our API. But most of our customers are not computer programmers and are instead utilising plugins and client software which may not take advantage of all our API responses or the developer of their software may have locked-in how the software reacts to most API responses and that may not suit your particular use case.

By moving this logic to the API side with custom rules it enables you to use even the most basic clients available that support our API and still make full use of all current and future features.

The UI we've designed for custom rules goes beyond just looking nice. It's highly functional with instant feedback letting you know if a condition or output modifier you've entered won't work. It can automatically disable rules if the rule isn't valid and let you know which part of a rule needs to be changed before it can be enabled.

We've also made it so you can drag and drop rules around enabling you to adjust the order in which rules are acted upon without the need to erase a rule just because it's in the wrong position.

Image description

Now you're probably thinking, great now the Whitelist and Blacklist features are going away. Well don't worry, that's not happening. We love those features and it makes it really easy for customers to do simple actions based on IP Addresses, IP Ranges and AS numbers. And in-fact the new rule system works fully in conjunction with those features, you can even make rules based on the result of those lists.

So this all sounds great right - but what does it cost! - I hear you saying. Well we're not charging extra for rules. Instead we're giving you a quantity of rules based on your current plan sizes. Customers on our free plan can enable three rules while our starter paid plan of $1.99 can enable six rules. Each plan can enable an additional three rules over the previous plan.

And to be clear, we're saying enable because all accounts regardless of plan size can create an unlimited amount of rules. You just can't enable more than your plan size allows. So you can create a lot of different rules for testing and see what works for you by toggling different rules on and off without needing to delete one rule to create another.

Each rule you create can have an unlimited amount of conditions and output modifiers and as I mentioned previously you can provide multiple values for each condition within a single rule, this is done by separating them by a comma. This means if you wanted to create a rule that applied to 20 different countries you could do that by simply listing 20 countries as a single value in a single condition. You do not need to create 20 different conditions or 20 different rules to accomplish that.

So that is the new custom rules feature. It's live right now for all customers within the dashboard and we would love to hear your feedback. The last thing to mention is this feature only works on our v2 API so if you've not made the switch from the v1 API now is the perfect time to do so!

Thanks for reading and have a great week.


Back