|
Post by Fed up with PC views only on Oct 5, 2020 21:34:13 GMT
On the news just now - the 'lost' tests/cases were due to PHE getting third parties (?) to send their data in using Excel files - which are limited to 65k records (I think) - anything after that just disappears!! Now - takes deep breath - some will no doubt be asking why they are using an old version of Excel. Which is a fairly valid question. But a far more important question is WHY THE FÚCK ARE THEY USING A BLOODY SPREADSHEET?? This is supposed to be a nationwide DATABASE, able to rapidly and reliably store and cross-reference hundreds of millions of pieces of data. For which, any remotely IT-aware person will tell you, you need a DATABASE - the clue is in the name - not a frigging bloody spreadsheet. I am almost beyond flabbergasted by this. It is, quite literally, unbelievable. Of course its unbelievable! Its not meant to be believable, just accepted by the populace through media brainwashing. JDL, are you starting to realise I was right all along and the whole Covid thing is an overhyped hoax to brainwash and enslave humanity by the top 1% who globally own everything from tech companies to pharma and banks (therefore governments) and want to re-shape the world to fit their view of how humanity needs to be controlled and regulated to ensure they sell their technology, drugs etc to ensure their super-rich lifestyles are maintained at our expense? I knew you'd get there eventually!
|
|
|
Post by jdh80 on Oct 6, 2020 7:59:57 GMT
On the news just now - the 'lost' tests/cases were due to PHE getting third parties (?) to send their data in using Excel files - which are limited to 65k records (I think) - anything after that just disappears!! Now - takes deep breath - some will no doubt be asking why they are using an old version of Excel. Which is a fairly valid question. But a far more important question is WHY THE FÚCK ARE THEY USING A BLOODY SPREADSHEET?? This is supposed to be a nationwide DATABASE, able to rapidly and reliably store and cross-reference hundreds of millions of pieces of data. For which, any remotely IT-aware person will tell you, you need a DATABASE - the clue is in the name - not a frigging bloody spreadsheet. I am almost beyond flabbergasted by this. It is, quite literally, unbelievable. You do realise that each area will be collating the data and submitting it on spreadsheets and then the spreadsheets are uploaded onto a central database where the data is then used to produce statistical analysis. Quite a lot of government projects are done this way, sometimes there will be the one central database that everything is uploaded onto and then there will be smaller adhoc versions of the database, that are used by each individual area to collect the information and then they export the data into either a txt file or excel to submit the results and these are then uploaded onto the central system.
|
|
|
Post by jdl on Oct 6, 2020 10:17:56 GMT
On the news just now - the 'lost' tests/cases were due to PHE getting third parties (?) to send their data in using Excel files - which are limited to 65k records (I think) - anything after that just disappears!! Now - takes deep breath - some will no doubt be asking why they are using an old version of Excel. Which is a fairly valid question. But a far more important question is WHY THE FÚCK ARE THEY USING A BLOODY SPREADSHEET?? This is supposed to be a nationwide DATABASE, able to rapidly and reliably store and cross-reference hundreds of millions of pieces of data. For which, any remotely IT-aware person will tell you, you need a DATABASE - the clue is in the name - not a frigging bloody spreadsheet. I am almost beyond flabbergasted by this. It is, quite literally, unbelievable. You do realise that each area will be collating the data and submitting it on spreadsheets and then the spreadsheets are uploaded onto a central database where the data is then used to produce statistical analysis. Quite a lot of government projects are done this way, sometimes there will be the one central database that everything is uploaded onto and then there will be smaller adhoc versions of the database, that are used by each individual area to collect the information and then they export the data into either a txt file or excel to submit the results and these are then uploaded onto the central system. Of course. But still a crazy way of doing it - as has been ably demonstrated. When I was in systems, we never touched spreadsheets, even for remote data collection (in fact especially for remote data collection!), they always cause problems. Any system should have its data capture aspects designed as part of the system design and they should use the same software, or, at least, software approved/designed by, and under the control of, the system designers. Otherwise you are at the mercy of the data input not being reliable or complete, or not matching the data in the database. And if your data input is compromised, so is your whole system. Not exactly difficult to do in these days of the internet.
|
|
|
Post by daveu on Oct 6, 2020 11:17:59 GMT
You do realise that each area will be collating the data and submitting it on spreadsheets and then the spreadsheets are uploaded onto a central database where the data is then used to produce statistical analysis. Quite a lot of government projects are done this way, sometimes there will be the one central database that everything is uploaded onto and then there will be smaller adhoc versions of the database, that are used by each individual area to collect the information and then they export the data into either a txt file or excel to submit the results and these are then uploaded onto the central system. Of course. But still a crazy way of doing it - as has been ably demonstrated. When I was in systems, we never touched spreadsheets, even for remote data collection (in fact especially for remote data collection!), they always cause problems. Any system should have its data capture aspects designed as part of the system design and they should use the same software, or, at least, software approved/designed by, and under the control of, the system designers. Otherwise you are at the mercy of the data input not being reliable or complete, or not matching the data in the database. And if your data input is compromised, so is your whole system. Not exactly difficult to do in these days of the internet. Unfortunately you can't always dictate what format you receive data in. I've developed systems that had to cater for multiple import formats due the diversity and limitations of the clients.
|
|
|
Post by nws on Oct 6, 2020 15:37:32 GMT
On the news just now - the 'lost' tests/cases were due to PHE getting third parties (?) to send their data in using Excel files - which are limited to 65k records (I think) - anything after that just disappears!! Now - takes deep breath - some will no doubt be asking why they are using an old version of Excel. Which is a fairly valid question. But a far more important question is WHY THE FÚCK ARE THEY USING A BLOODY SPREADSHEET?? This is supposed to be a nationwide DATABASE, able to rapidly and reliably store and cross-reference hundreds of millions of pieces of data. For which, any remotely IT-aware person will tell you, you need a DATABASE - the clue is in the name - not a frigging bloody spreadsheet. I am almost beyond flabbergasted by this. It is, quite literally, unbelievable. Of course its unbelievable! Its not meant to be believable, just accepted by the populace through media brainwashing. JDL, are you starting to realise I was right all along and the whole Covid thing is an overhyped hoax to brainwash and enslave humanity by the top 1% who globally own everything from tech companies to pharma and banks (therefore governments) and want to re-shape the world to fit their view of how humanity needs to be controlled and regulated to ensure they sell their technology, drugs etc to ensure their super-rich lifestyles are maintained at our expense? I knew you'd get there eventually! So you argued to vote in a government that you have long believed is enacting policies that benefit the top 1% at the expense of everyone else. I'm pleased you have started to see the light
|
|
|
Post by jdh80 on Oct 7, 2020 0:24:39 GMT
You do realise that each area will be collating the data and submitting it on spreadsheets and then the spreadsheets are uploaded onto a central database where the data is then used to produce statistical analysis. Quite a lot of government projects are done this way, sometimes there will be the one central database that everything is uploaded onto and then there will be smaller adhoc versions of the database, that are used by each individual area to collect the information and then they export the data into either a txt file or excel to submit the results and these are then uploaded onto the central system. Of course. But still a crazy way of doing it - as has been ably demonstrated. When I was in systems, we never touched spreadsheets, even for remote data collection (in fact especially for remote data collection!), they always cause problems. Any system should have its data capture aspects designed as part of the system design and they should use the same software, or, at least, software approved/designed by, and under the control of, the system designers. Otherwise you are at the mercy of the data input not being reliable or complete, or not matching the data in the database. And if your data input is compromised, so is your whole system. Not exactly difficult to do in these days of the internet. Which is another reason why you have stand alone systems in each of the areas uploading data to a central team, as the central team can check the data for completeness and if it's incomplete it is sent back and not uploaded onto the central system to avoid compromising the rest of the data.
|
|
|
Post by sword65 on Oct 7, 2020 0:49:26 GMT
Zzzźzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz!
|
|
|
Post by nws on Oct 7, 2020 23:37:08 GMT
This government should use pigeon post as they are mostly pigeon brained
|
|
|
Post by jdl on Oct 8, 2020 9:00:56 GMT
Of course. But still a crazy way of doing it - as has been ably demonstrated. When I was in systems, we never touched spreadsheets, even for remote data collection (in fact especially for remote data collection!), they always cause problems. Any system should have its data capture aspects designed as part of the system design and they should use the same software, or, at least, software approved/designed by, and under the control of, the system designers. Otherwise you are at the mercy of the data input not being reliable or complete, or not matching the data in the database. And if your data input is compromised, so is your whole system. Not exactly difficult to do in these days of the internet. Which is another reason why you have stand alone systems in each of the areas uploading data to a central team, as the central team can check the data for completeness and if it's incomplete it is sent back and not uploaded onto the central system to avoid compromising the rest of the data. But you never use spreadsheets. Apart from the fact that you have no control over how the data is entered, it can too easily be deleted or amended. And their is absolutely no audit trail. A system is only valid if it's data is known to be valid. Any system that uses input via spreadsheets cannot be relied upon to be either complete or accurate. As we have seen!
|
|
|
Post by Tstone on Oct 8, 2020 9:13:04 GMT
Which is another reason why you have stand alone systems in each of the areas uploading data to a central team, as the central team can check the data for completeness and if it's incomplete it is sent back and not uploaded onto the central system to avoid compromising the rest of the data. But you never use spreadsheets. Apart from the fact that you have no control over how the data is entered, it can too easily be deleted or amended. And their is absolutely no audit trail. A system is only valid if it's data is known to be valid. Any system that uses input via spreadsheets cannot be relied upon to be either complete or accurate. As we have seen! Spreadsheets (Excel at least) have the functionality to control the type of data input into a cell, as well as checking if data is within certain valid ranges.
Parts of the spreadsheet, also down to individual cell e.g. headings or calculations, can be locked from user (not programmer) input.
And there are loads of functionality I never use.
|
|
|
Post by jdl on Oct 8, 2020 9:24:47 GMT
But you never use spreadsheets. Apart from the fact that you have no control over how the data is entered, it can too easily be deleted or amended. And their is absolutely no audit trail. A system is only valid if it's data is known to be valid. Any system that uses input via spreadsheets cannot be relied upon to be either complete or accurate. As we have seen! Spreadsheets (Excel at least) have the functionality to control the type of data input into a cell, as well as checking if data is within certain valid ranges.
Parts of the spreadsheet, also down to individual cell e.g. headings or calculations, can be locked from user (not programmer) input.
And there are loads of functionality I never use.
From a system design point of view, I'm afraid they are a nightmare. You just can't trust the data to be complete or valid. And users love imagining they are programmers and can't resist writing bits of 'code' - which never works they way they expect it to! There was an interesting piece on this on Radio 4's More or Less this week - www.bbc.co.uk/sounds/play/m000n4vq. It's worth a listen (it's the first item and only lasts a few minutes).
|
|
|
Post by daveu on Oct 8, 2020 10:04:39 GMT
Spreadsheets (Excel at least) have the functionality to control the type of data input into a cell, as well as checking if data is within certain valid ranges.
Parts of the spreadsheet, also down to individual cell e.g. headings or calculations, can be locked from user (not programmer) input.
And there are loads of functionality I never use.
From a system design point of view, I'm afraid they are a nightmare. You just can't trust the data to be complete or valid. And users love imagining they are programmers and can't resist writing bits of 'code' - which never works they way they expect it to! There was an interesting piece on this on Radio 4's More or Less this week - www.bbc.co.uk/sounds/play/m000n4vq. It's worth a listen (it's the first item and only lasts a few minutes). Whatever your view of spreadsheets, sometimes you have to be driven by the needs/desires of the client. I've spend 40 years working as a programmer and I've had to write imports for all kinds of weird file formats, including strange systems written by small business owners who thought it a good idea to be their own IT department. Believe me, some of the shit data I've had to import over the years makes Excel look like a genuis option.
|
|