+ Post New Thread
Page 1 of 6 12345 ... LastLast
Results 1 to 15 of 89
MIS Systems Thread, The Analysis of Assessment Data in your school in Technical; Hi all I’m an analyst working for Bromcom Computers. I have been assigned a task to spec’ a new assessment ...
  1. #1

    Join Date
    Jul 2008
    Location
    London
    Posts
    4
    Thank Post
    0
    Thanked 0 Times in 0 Posts
    Rep Power
    0

    The Analysis of Assessment Data in your school

    Hi all

    I’m an analyst working for Bromcom Computers. I have been assigned a task to spec’ a new assessment analysis software module.

    The primary objective of the software module is to give schools a powerful but easy to use software analysis tool that can quickly collate, filter, sort, plot/chart and tabulate recorded assessment data into information that schools can use to determine and track performance from the student level > subject level > department level > school level.

    Now, I confess I am a complete newbie when it comes to analysis of assessment data and my present knowledge and understanding is limited. Therefore, I would appreciate any information and guidance on this topic. I am interested in both primary and secondary schools’ responses. Specifically:

    1. What kind of assessment analysis do you presently do in your school?

    For example do you produce: Student/Subject Residuals, % of students achieving certain grade ranges for the core subjects?

    2. What kind of trend/predicted analysis do you do?

    3. How do you make use of “Value added” data such as FFT, Yellis, MidYis, Raiseonline?

    4. How much of this analysis is done in your MIS?

    5. How much of the analysis done outside of your MIS – say in Excel?

    6. How to you choose to present your data: Tables? Bar graphs? Line graphs? Colour coded conditional formatting?

    7. How often is this analysis done?

    8. Who does this analysis in your school?

    9. What problems/difficulties do you encounter with your current analysis process?

    10. What features and functions would you like to see in assessment analysis module?

    Please feel free to add any other comments.

    Thanks in advance.

  2. #2

    Join Date
    Dec 2009
    Posts
    59
    Thank Post
    2
    Thanked 8 Times in 6 Posts
    Rep Power
    14
    Hopefully you will get a lot of input Kozan - I'd add more if i had the time, but as a quick response:

    from a secondary school perspective, if the module you developed would support the type of usability and presentation being offered currently by external commercial solutions such as 4Matrix and Sisra Online, to review current progress as well as final results data, then you would win over school leadership teams no problem whatsoever!

    By implication, at KS4 this means the assessment module should allow users to define means of recording grades against elements that have their (DCSF) performance measure contributions built in, so that the analysis part of the system can do all the hard work for the user, and allow the level of top level decision making for intervention etc that the above examples are currently providing.

    I have always felt that the MIS providers should be delivering this now, but I'm not sure any of them get anywhere close - if you look at it from the point of view of the amount of work and skills required to set-up analysis and then the user interfaces provided to exploit the data.

    Monitoring progress at KS4 is a good example I feel - it simply shouldn't be necessary to have to fashion a recording mechanism with an MIS, build in intelligence by using whatever formula mechanism to summarise key criteria by (by which schools are judged), and then fashion a whole host of different "views" of that data to provide a way for decisions to be made from it.

    Catering for all possible types of qualifications that schools may be delivering is a big ask, but I don't see it as an unreasonable expectation from an MIS system, when there are MIS systems out there that support this level of "intelligent" analysis for summer exam results.

    Hope that helps.

  3. #3

    Join Date
    Jul 2008
    Location
    London
    Posts
    4
    Thank Post
    0
    Thanked 0 Times in 0 Posts
    Rep Power
    0
    Thank you Resin1 - most useful. I had been looking at SISRA online prior to my post - but I don't know much about 4matrix. I'll check it out.

    Based on my research thus far, it seems to me that schools are looking for and would value a graphical/visual approach to data analysis - since most people receive 80% of thier information visually. Would you agree?

    I do appriciate your time - but if you have another spare five minutes - please could you expound on the following statement:

    "the assessment module should allow users to define means of recording grades against elements that have their (DCSF) performance measure contributions built in, so that the analysis part of the system can do all the hard work for the user, and allow the level of top level decision making for intervention etc that the above examples are currently providing."

    Thanks in advance.

  4. #4

    dhicks's Avatar
    Join Date
    Aug 2005
    Location
    Knightsbridge
    Posts
    5,618
    Thank Post
    1,230
    Thanked 776 Times in 673 Posts
    Rep Power
    235
    Quote Originally Posted by Kozan View Post
    What kind of assessment analysis do you presently do in your school?
    Basic percentages-of-grades.

    What kind of trend/predicted analysis do you do?
    I forget the exact name and algorithm at the moment, but something provided by one of the members of staff (who is a good statistician and knows what she's doing).

    How do you make use of “Value added” data such as FFT, Yellis, MidYis, Raiseonline?
    I'm not sure.

    How much of this analysis is done in your MIS?
    None.

    How much of the analysis done outside of your MIS – say in Excel?
    The analysis is currently done by a set of Python scripts that takes in a set of raw figures and produces a bunch of tables and charts. This is flexible but not very maintainable (i.e. I know how it works, no-one else does). Unfortunatly, none of our teaching staff have enough knoweldge of Excel to be able to use it.

    How to you choose to present your data: Tables? Bar graphs? Line graphs? Colour coded conditional formatting?
    Tables, assorted charts.

    How often is this analysis done?
    Once a year, which much wailing and gnashing of teeth, although given more data updates the script could easily be run every night or however often is required.

    Who does this analysis in your school?
    Me.

    What problems/difficulties do you encounter with your current analysis process?
    Teaching and admin staff don't know enough about statistics to be able to properly articulate what it is they want to show, and if they did then they lack the IT skills to convert that articulation in to tables or charts. It should be perfectly possible to knock something together with Excel but they simply do not have the skills to do that.

    What features and functions would you like to see in assessment analysis module?
    It needs to produce "nice looking" charts that we can use for marketing purposes. It needs to be simple to use, with no fuss, for people with few basic IT skills.

    --
    David Hicks
    Last edited by dhicks; 30th March 2010 at 05:56 PM.

  5. #5

    Join Date
    Dec 2009
    Posts
    59
    Thank Post
    2
    Thanked 8 Times in 6 Posts
    Rep Power
    14
    ok Kozan - quickly ;-)

    yes - visual, visual visual

    what I mean by that statement, is that I'd want to be able to set-up a progress tracking system where for example, I say for this subject we need to collect predicted grades that are full GCSE (A*, A, B, C etc) and I am able to chose an element that allows those types of grades to be input and within the system the element is classed as "1 whole GCSE" or in other words 20% contribution to Level 2 performance measure threshold (5+ A*-C) and that a A* is worth 58 QCA points. Similarly, I could assign a GCSE grade collection element for a GCSE short course, and the system would know it is 1/2 a GCSE/10% contribution/A*=29 etc. Or for an OCR National Level 2 Certificate, I could collect Pass/Merit/Distinction and it would know these are 4 GCSE/80% contribution/Distinction=220 points etc.

    see Home Page - National Database of Accredited Qualifications if you are not familiar with all that stuff (search for a qualification and look at its performance measures).

    Although I haven't used either of the those systems I mentioned, I believe what they provide is a mechanism to map subject name/raw grade data collected via MIS or other, to an internal qualification database that then means they can analyse that raw data, present it meaningfully and allow drill down by users for decision making, in a context within which those users are being judged by external parties.

    Although I'm sure there will be a place in the market for them for some time to come and I'm not wishing them out of business (apologies if any of their reps look in here) but it would seem to me that an MIS should be able to offer something for assessment/progress monitoring that is designed and fit for the purpose(s) of a large number of users out there, rather than delivering something that is generic and may allow all to do something, but doesnt really fit a specific purpose.

  6. #6

    Join Date
    Sep 2006
    Location
    London
    Posts
    1,310
    Thank Post
    35
    Thanked 349 Times in 235 Posts
    Rep Power
    77
    I'm sure all these companies will appreciate you all helping someone copy their ideas!

  7. #7

    vikpaw's Avatar
    Join Date
    Sep 2006
    Location
    Saudi Arabia
    Posts
    5,848
    Thank Post
    671
    Thanked 1,383 Times in 1,145 Posts
    Rep Power
    350
    hi kozan
    worth making the module talk easily with external systems, and have flexible input and output of data. e.g. raiseonline, and fft data, both heavily used by ofsted.
    possibly worth contacting those bodies as well.
    there's also another body to do with ks2/3 grades that i made contact with some time ago, i forget the name, i think it's the people that send the CAT scores out, worth talking to them, the developer was most helpful to me, once i'd socially engineered his private mobile that is!
    good luck.

  8. #8

    vikpaw's Avatar
    Join Date
    Sep 2006
    Location
    Saudi Arabia
    Posts
    5,848
    Thank Post
    671
    Thanked 1,383 Times in 1,145 Posts
    Rep Power
    350
    Quote Originally Posted by PhilNeal View Post
    I'm sure all these companies will appreciate you all helping someone copy their ideas!
    Can't harm to see what's on the market to see what you can Discover

  9. #9
    skunk's Avatar
    Join Date
    Mar 2006
    Location
    North West
    Posts
    311
    Thank Post
    88
    Thanked 40 Times in 33 Posts
    Rep Power
    29
    Quote Originally Posted by PhilNeal View Post
    I'm sure all these companies will appreciate you all helping someone copy their ideas!
    Someone needs to pull of the ideas from these disparate systems and put them all in one system that schools can use effectivley, we cannot afford five systems that each do a little bit well. My MIS's analysis suite currently does not do half (or even a tenth if I am honest) the things I would wish, but I would be happy to swap if someone were to write one that did all the great stuff I have to do by hand in excel currently.

    Students matter, products do not.

  10. #10

    Join Date
    May 2009
    Location
    Sheffield
    Posts
    243
    Thank Post
    22
    Thanked 35 Times in 26 Posts
    Rep Power
    20
    Quote Originally Posted by PhilNeal View Post
    I'm sure all these companies will appreciate you all helping someone copy their ideas!
    That occurred to me too, Phil. Certainly, the one area where most MIS software is lacking is in providing ready-made analyses of the kind that schools actually want; marketing materials from the various vendors (Bromcom and Capita included!), alongside blanket questionnaires like this, suggest that this is unlikely to change in the immediate future.

    I don't really want to write a full analysis suite spec for free, but I will give some tips:

    • Have a read through the literature - there are many guides online to analysing assessment data available, which will give you an idea of typical foci of interest. The national strategies sites, forums such as this one, LA support sites, etc, have guidance on how to measure performance, and what to look for.
    • Talk to some schools, i.e. go and visit them and see what they do - then you can think about how to solve the problems they're having to solve themselves at the moment
    • Look through the feature requests for products like TeachersWebFolder - would any of these fit your brief?
    • Have a read through DCSF and Ofsted guidance on how to assess the effectiveness of schools and LAs - this will give you an idea of the kind of questions schools will need to answer


    It's going to be difficult to make anything easy for schools for as long as MIS software allows users to store assessment data the way they want. A lot of the time, schools tend to be very focussed on colour-coded lists of students: a quick search online will show you what they're asking about!

    If this is a sanctioned project at Bromcom, you may want to look at getting in a few consultants (some who are "data-focussed" and some who are "anti-data") to discuss what they think would be helpful. Is this a Bromcom thing? I would imagine the firm is big enough to pay a team to develop a solution, and not just to let a new analyst "have a go" based on what they can find in a discussion forum...

    Overall, the kind of questions you need to answer are:
    • What's going wrong, and for which subject/students/groups of students?
    • What's working, and (again) where is it working?
    • How did we do in the past? How will we do in the future?
    • Where do we need to improve in order to hit XYZ threshold target?

  11. #11

    Join Date
    Dec 2009
    Posts
    59
    Thank Post
    2
    Thanked 8 Times in 6 Posts
    Rep Power
    14
    The above comments made me smile - especially Vik's - nice one.

    Certainly dont see the need to defend my self, but I'll just say that any vendor worth their salt should be constantly reviewing and researching to see what other vendors or complimentary solutions are up to, in order to either stay one step ahead of the competition or try to at least deliver something that is competitive. Anything stated earlier could be gleaned within a few minutes perusal of their web-sites, which I'm sure, Kozan's marketing or product management colleagues would hit sooner or later if he didn't himself.

    To me they were just examples of what users might demand. Heaven forbid that one vendor might be influenced by another commercial offering

  12. #12
    Matthews's Avatar
    Join Date
    Dec 2009
    Location
    Warwickshire
    Posts
    8
    Thank Post
    3
    Thanked 0 Times in 0 Posts
    Rep Power
    0
    Hi

    I am currently looking for some software that can produce this kind of analysis. I currently manually create analysis from Assessment Manager7 and spreadsheets. We are a large school and this can be an extremely time consuming process, especially now that we have to produce minority analysis along side the normal analysis.

    We need software that can tell us:

    Percentage A*-C in KS4 & 5 in each subject for Assessments, Targets & FFT or Alis
    Percenatge 5+ & 6+ in Ks3 Subjects for Assessments, Targets & FFT

    This data also needs further analysis to tell us how many of these are Girls,Boys,SEN,FSM,EAL & Ethnic Minorty Students.

    Currently the process i'm using takes me two weeks to collate.

    I'm sure i'm probably doing it in a much longer way than needed.

    Can anyone tell me what software (if any) they use? Or how they use SIMs to generate such analysis that may cut my time spent doing our termly analysis.

    Any help is appreciated.

    Kelly

  13. #13
    Tyiell's Avatar
    Join Date
    Apr 2009
    Location
    Like everyone in IT - I'm omnipresent...
    Posts
    373
    Thank Post
    147
    Thanked 67 Times in 44 Posts
    Rep Power
    24
    @Mathews: Our brilliant data manager has built a phenomenal spreadsheet that does this and more - it's mind-boggling to behold, but she can more or less export the data from sims and copy it in and boom - it all works. I can ask her if she will let me post it for you if you like, but be warned - it's MAAASSSSIVVE (and pink, she likes all her spreadsheets to be pink!!).

  14. Thanks to Tyiell from:

    Matthews (19th April 2010)

  15. #14
    Matthews's Avatar
    Join Date
    Dec 2009
    Location
    Warwickshire
    Posts
    8
    Thank Post
    3
    Thanked 0 Times in 0 Posts
    Rep Power
    0
    lol... Tyiell

    I have tried to do a spreadsheet to house analysis and i can get one to do KS3 as our students in KS3 all do the same subjects. It gets more complicated in KS4 due to their option choices etc...

    But yes please if you could post it or send me a copy i'd be enternally grateful!!!

    I would ofcourse want to pay her (in biccies) for her trouble

  16. #15

    Join Date
    Apr 2006
    Location
    West Midlands
    Posts
    314
    Thank Post
    29
    Thanked 19 Times in 18 Posts
    Rep Power
    20
    @Mathews - can't you use analysis and trend tracking tools in AM7?

SHARE:
+ Post New Thread
Page 1 of 6 12345 ... LastLast

Similar Threads

  1. Assessment Data(CMIS)
    By Kained in forum How do you do....it?
    Replies: 7
    Last Post: 2nd April 2011, 12:47 PM
  2. Replies: 0
    Last Post: 26th March 2010, 12:53 PM
  3. Encryption of school data
    By Sylv3r in forum How do you do....it?
    Replies: 27
    Last Post: 12th May 2009, 09:10 AM
  4. Promissor - data collection & analysis
    By j17sparky in forum Educational Software
    Replies: 3
    Last Post: 16th May 2008, 02:03 PM
  5. How does your school use assessment manager?
    By thegrassisgreener in forum MIS Systems
    Replies: 9
    Last Post: 19th December 2007, 10:20 AM

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •