I’m writing a web crawler in Java. The target is a website written in react. I use selenium to render the page to a DOM tree:
HtmlUnitDriver unitDriver = new HtmlUnitDriver(); unitDriver.get("https://www.instagram.com/9gag/?hl=en"); WebElement body = unitDriver.findElement(By.tagName("body"));
What I get for body is:
Which is almost nothing. Does selenium render react pages ? I also tried this on google.com and it worked fine.