Document Actions
Monitoring for Day Centres
Monitoring the performance of your day centre is vital to proving its value to funders, and to ascertaining that you are in fact achieving what you set out to.
On this page:
Introduction | Indicators | Being Flexible | Data Collection Methods | Ethical Issues in Monitoring
Jump to pages on:
- Getting Started
- Special feature: Monitoring for LAA indicators
- Evaluation
- Further reading and resources
Introduction
Once you know what you are trying to achieve and how you intend to achieve it, you must think about how you are going to measure your success – this is what is known as monitoring. The theory of change model (see: Getting Started is in fact a very basic monitoring and evaluation framework: you need to record data about your resources, outputs and outcomes in order to later evaluate your success against your objectives. In order to fully evaluate your day centre, you will need to collect output and outcome data about your service users and the service themselves, and you will also need to look for data from outside of your service, for example about your local area, that you can compare your results to. The tricky bit, however, is knowing what data you need to effectively monitor your centre. This section of the website aims to give you the tools you need to set up an efficient monitoring framework for your day centre.
Indicators
The first step in setting up a monitoring framework is to choose your indicators. Indicators are the information you need to collect about your service and service users depending on what you want to show.
There are some things that every day centre should know about its services and its service-users because it will need to show that people are using the centre, and that those people are indeed the group that you are trying to target with your interventions. You may also want to collect additional profile information so that you will know if you need to tailor your services to any particular group. This initial data collection about people accessing your service will be your baseline data to compare to your outcomes – you need to be able to know where people started in order to show that they have made progress. Essential baseline information includes:
Information about service users:
- Biodata: age, sex, ethnic origin, family status. If you are not targeting a particular group (e.g. ethnic minorities, single people, older people), this information is still important in order to show that your service is equally accessible to all. If your service is equally accessible, then your biodata statistics should be roughly proportionate to statistics in your area.
- Housing status: the core categories are rough sleeping, emergency accommodation, hostel, temporary accommodation (statutory), long-term supported accommodation, and independent living, but this could be extended to include council flat, private rented, home-owner, squatting, B&B, etc. Furthermore, housing status should be tracked and updated.
- Specific needs: ask about substance use, mental health, physical health, learning disabilities, care leavers.
Information about services:
- Number of people using the centre per day.
- Number of people participating in specific interventions (the member of staff running each activity should record this information, as well as which clients are participating so that their biodata can be compiled per activity).
- Outcomes for the service: if the service has an outcome tied to it, for example a qualification, then these need to be recorded.
Really, the more detailed and comprehensive information you have, the better. If humans by nature were happy to share private information about themselves and to spend long periods of time filling out surveys, then it would be a good idea to collect as much information about them as you possibly could. However, since that is generally not the case, one has to balance the benefit to the client of getting enough data to be able to give them the most appropriate and effective service with the potential harm to the client of asking too many or too detailed questions. So, you need to choose your indicators carefully to relate to your objectives. Go back and look at your theory of change model. If your theory is that one way to achieve your objective of, for example, helping vulnerable people into independent living, is to provide housing advice and referral, then you will want your indicators to tell you how successful you are at advising and referring people, and how successful this is at moving them towards independent living. Your indicators will then be:
- number of people receiving advice.
- number of referrals made.
- number of people making a planned move into more independent living.
- number of people achieving independent living.
Your baseline data will be:
- accommodation status before receiving advice.
Being Flexible
Data collection should also be flexible. If you notice an emerging trend, for example if you think you are seeing an increasing proportion of LGBT clients, then you might want to start collecting additional data around this. For example, do a snapshot survey on any given day to see how many of your service’s clients are LGBT and find out why they are using the centre. Repeat the exercise in 6 months and see what has changed. If you realise that people would benefit from a LGBT-specialised project worker or could be referred to a different organisation that is LGBT-specific already, then you can use the data you have collected to apply for funding for a new project worker or to form a partnership with another organisation.
Data Collection Methods
How you collect information will depend on the dynamics and resources of your day centre. For day centres that have the capacity, it’s a good idea to use a soft outcomes monitoring system in order to show distance travelled. For day centres that are a little shorter of resources, snapshot surveys or interviews can often provide the data that you need.
Basic Data Collection Tips
Be systematic: You will need to have a coherent strategy by which to collect and store information that is suitable to your resources. For example, for profile information, ideally you would be able to collect information from every client entering the day centre as part of a needs assessment. However, if your day centre operates a more open door policy, you might only collect detailed information from clients who participate in interventions.
Get staff onboard: Make sure your staff know what information they should be collecting and also that they know the purpose of data collection. If staff are not aware of the importance of data collection and monitoring and evaluation generally, then they might not see it as a priority and not accurately or consistently record data.
Store and sort your data: It will be important to create a database in which to store and manipulate the data that you collect. A variety of custom-made databases are available, but if you are short of resources, a simple Excel spreadsheet can be very effective. Remember to enter your data into the database on a regular basis so that you can stay on top of trends and outcomes and so that it doesn’t become a mammoth task at the end of the year.
Snapshot Surveys
Snapshot surveys are particularly good for day centres who are short of resources. A snapshot survey is basically a survey whose sample is a 'snapshot' at a particular point in time - for day centres, this will normally be the clientele on a random day during the year. As long is there is nothing out of the ordinary about that particular day, then it is reasonable to claim that the clientele interviewed for the survey is representative of the day centres total client base. If possible, it is advisable to conduct more than one snapshot survey (e.g. one per season) to make the data collected even more representative and reliable.
There are several advantages to snapshot surveys. They are not too invasive for your clientele since they are only conducted infrequently, and the more targeted they are, the fewer questions you will have to ask. They also do not overly tax staff resources, since, again, they are conducted on only a few days a year. Finally, they can provide you with a 360 degree perspecitive of an issue in your service, since every client present can participate.
Snapshot surveys work best if the list of questions is short and targeted - so you should know what you are trying to monitor. For example, if you don't have the resources to record the biodata of every client using your centre, you could use a snapshot survey to get an idea of who comes to the centre. It would then be useful to compare this to information about specific clients participating in interventions. For example, you might find that a portion of service users who are privately housed are coming to the centre only for cheap or free meals but not participating in any interventions. It would then be appropriate to approach some these individuals and to conduct more detailed interviews with them to find out if they require additional support (e.g. budget management). Or if they are not your target population, then you might want to consider amending your service provision to make it more efficient (e.g. start charging for meals, or providing free meals only to those who participate in an intervention).
Remember, a snapshot survey is a tool to collect supplementary data about your service - it should not replace systematic data collection about your service provision and service users and it cannot be used for outcomes monitoring.
Interviews
Interviews are a good way to get data about how your services are making a change. For example, if you interview a representative sample of your clients and 71% of them agree that meaningful occupation activities at the day centre have helped them in controlling their substance use, that is an excellent bit of supporting data to show that your intervention is helping to achieve a change that contributes to your objective. However, interviewing requires greater skill and knowledge to conduct properly than a snapshot survey, and it may be adviseable to ask an external organisation or a research student to conduct interviews for you to make sure that they are credible. A day centre client, after all, may feel pressured to give a positive response to a member of staff.
Soft outcomes monitoring
Soft outcomes are the results of interventions that do not translate into countable results (called hard outcomes) – e.g. number of qualifications awarded, number of people entering full-time employment. These are often qualitative improvements in areas such as self-esteem, presentation, and self-evaluated physical and mental health. For day centres, these can be particularly important, since the some of the principal aims of the service are likely to be in soft outcomes areas, the idea being to give people the confidence and self-efficacy to make their own changes in life. Day centres are often responsible for helping people make the first steps before keying them into more specific services which will deliver the hard outcomes. For example, a person might start using a day centre wet garden, and interact with centre staff for months receiving advice and support to control their drinking before being able to approach a hostel or training programme to continue their journey. The only hard outcome a day centre can record is a referral – which does not quite capture the extent of the work done with the client. If, on the other hand, the day centre has recorded the individual’s progress on soft outcomes, including alcohol consumption and how that person is feeling about their drinking (perhaps moving from the pre-contemplation stage to the contemplation or action stage), this will not only help them to judge when the appropriate time to make a referral might be, but will also help them to demonstrate the work that they have put into helping that person begin their journey out of homelessness. There are many tools available to monitor soft outcomes, and many day centres have designed their own. One that is easily available and well-supported is the Outcomes Star. For more information, go to http://www.homelessoutcomes.org.uk/.
Ethical Issues in Monitoring
It is highly important to make sure that your monitoring processes are ethical, especially when collecting data from vulnerable people. For example, confidentiality and consent are two critical issues that day centres must consider when asking their service users for information. The issues are too many and too complex to be fully considered here, but they are widely addressed by research associations and universities nationally and we recommend that you take a look at one or two codes of ethics.
You can download the SRA ethical guidelines here.
See the special feature on monitoring for LAA indicators or continue to Evaluation
Back to Funding for Day Centres