You can get JSON with cURL by sending a GET request with the -H "Accept: application/json" option. Here is a sample command that sends a GET request to our hosted…

To send Basic Auth credentials using cURL you need to use the -u option with "login:password" where "login" and "password" are your credentials.Here is a sample command that sends a…

To follow redirect using cURL you need to use the -L option. Here is a sample command that sends a GET request to our hosted version of HTTPBin and follows…

You can send JSON with a POST request using cURL using the -X option with POST and the -d option (data).Here is a sample command that sends a POST request…

You can select elements by class in XPath by using the contains(@class, "class-name") or @class="class-name" expressions.The first expression will match any element that contains class-name. Even if the element has additional…

You can ignore non-HTML URLs when web crawling via two methods.Check the URL suffix for unwanted file extensionsHere is some sample code that filters out image file URLs based on…

You can parse dynamic CSS classes using text-based XPath matching. Here is a short example of what HTML with dynamic CSS classes might look like:<div class="dd"> <h1 class="aa">Product Detailsh1> <div…

You can get XML with cURL by sending a GET request with the -H "Accept: application/xml" option. Here is a sample command that sends a GET request to our hosted…

To ignore invalid and self-signed certificates using cURL you need to use the -k option. Here is a sample command that sends a GET request to our hosted version of…

You can select elements by text in XPath by using the contains(text(), "Text string") or text()="Text string" expression.The first expression will match any element that contains the "Text string" sub-string.…