Answering this question is surprisingly hard, because there was an evoluation available from mechanical calculators to programmable calcuulator into modern computing. A machine which comes close to a modern computer is the PDP-1 released in 1959. This machine was equipped with a RAM memory, was used to run programs and there was even a monitor plugged into the device. According to Wikipedia, the pdp-1 was equipped with only 12kb of RAM which is smaller what the VC-20 has to offer.
Of course, there were many machines available before the pdp-1 but they can't be called a computer. If a machine has a main memory of only 1 kb or smaller its not possible to store a program into the machine. Also it can't be used for any practical application.
Early calculating machines like Mark I (1944) and the Zuse Z3 (1941) have more in common with a pocket calculator like TI-58 (1977). The only purpose of Zuse Z3 and Mark I was to calculate numbers in a predefined sequence. Their usefulness was little or even not there. The reason why these machines were built was to get a better understanding about the Von neumann architecture.
In parallel to the development of early calculating machines there was a shortage in main memory. Early attempts like magnetic core memory and williams tube (both invented in the 1950s) were very slow and had a small amount of memory. So the engineers of the 1950s were not able to build more powerful computers.
What was available until the advent of the pdp-1 were endless amount of mechanical calculators. Most of them were not controlled by punch cards, but larger models were equipped with this capabilities. The mechanical calculators until the 1950s were used for all the calculating tasks in buisness and science and were seen as highly advanced technology.
But let us go back to the initial question about the very first computer worldwide. What we can say for sure is, that in the 1940s in many places at the same time the first program controlled machines were built. Most of them with relais and vaccum tubes. It seems, that the building these machines was mostly a learning by doing task. At first the engineers built only the arithmic unit and then they have added a memory and other components. If one machine was built, the next model was planned and this resulted into a fast development which has evolved into today's computing industry.
According to the precise timeline, the zuse Z1 was the first computer in the world which was built in 1937. But only 7 years later, the Mark I (1944) was released in the U.S. and the machine was much better designed than any Zuse machine. The reason why especially early computing machines from the 1940s are interesting for today's engineers is because the improvement was very fast. It took only months until the next improved version of a calculating machine was built which was based on a complete different hardware. This fast development is surprising because the decades before during the manufactoring of mechanical calculators there was only a little improvement visible. A calculating machine from the 1870 looks the same like a calculater from the 1920s.