Was it Hoover's administration that caused the Great Depression that affected the USA and the rest of the world?

No, Hoover's policies were not the cause of the Great Depression---though they may have made it worse.

The consensus among economists and historians as to the cause of the Great Depression is that the stock market panic of 1929 triggered a financial crisis which threw hundreds of banks into insolvency. Droughts also contributed by reducing food production and raising food prices.

The response---or lack thereof---was crucial to what made the Great Depression so great. Banks were allowed to fail; the FDIC did not yet exist to insure banks against failure. This created a cascade through the financial system, where suddenly billions of dollars effectively evaporated from existence as the collapse of one bank which owed to another bank collapsed the next bank, and so on. There was effectively far less money supply than there had been previously, so there was not enough money to purchase the products being sold. Because the US was still on the classical gold standard, the Federal Reserve had very little power to expand the money supply.

This led to deflation, which made debts even worse, and also triggered layoffs because workers are loathe to accept lower nominal wages even if their inflation-adjusted wage has not changed.

Hoover was also quite conservative in terms of his fiscal policy response; he could have initiated huge government investment projects to employ workers and increase spending---as FDR would later do in the New Deal---but he largely avoided doing so, believing that the market would simply correct itself in due time and any such intervention would do more harm than good. He also adamantly refused to engage in deficit spending, even though a depression is exactly the time when deficit spending is necessary.

Hoover cannot be blamed for the stock market panic itself, and of course he cannot be blamed for droughts. So in that sense he did not cause the Great Depression. But his response was far too passive and ineffectual, and resulted in a much more prolonged and severe depression than would have occurred under better fiscal and monetary policy.