They’re likely blocking user agents too, which I think also doesn’t have legal enforcement (as in DuckDuckGo can just use “Google” unless they said otherwise.
LinkedIn tried blocking scraping that way but as long as the scraping isn’t burdensome it’s basically legal but you can still be bound by TOS and civil claims
They’re likely blocking user agents too, which I think also doesn’t have legal enforcement (as in DuckDuckGo can just use “Google” unless they said otherwise.
LinkedIn tried blocking scraping that way but as long as the scraping isn’t burdensome it’s basically legal but you can still be bound by TOS and civil claims
https://natlawreview.com/article/hiq-and-linkedin-reach-proposed-settlement-landmark-scraping-case