Security Metrics

A metric is a quantitative measurement that can be interpreted in the context of a series of previous or equivalent measurements. Metrics are necessary to show how security activity contributes directly to security goals; measure how changes in a process contribute to security goals; detect significant anomalies in processes and inform decisions to fix or improve processes. Good management metrics are said to be S.M.A.R.T:

• Specific: The metric is relevant to the process being measured.
• Measurable: Metric measurement is feasible with reasonable cost.
• Actionable: It is possible to act on the process to improve the metric.
• Relevant: Improvements in the metric meaningfully enhances the contribution of the process towards the goals of the management system.
• Timely: The metric measurement is fast enough for being used effectively.

Metrics are fully defined by the following items:

• Name of the metric;
• Description of what is measured;
• How is the metric measured;
• How often is the measurement taken;
• How are the thresholds calculated;
• Range of values considered normal for the metric;
• Best possible value of the metric;
• Units of measurement.

Security Metrics are difficult to come by
Unfortunately, it is not easy to find metrics for security goals like security, trust and confidence. The main reason is that security goals are “negative deliverables”. The absence of incidents for an extended period of time leads to think that we are safe. If you live in a town where neither you nor anyone you know has ever been robbed, you feel safe. Incidents prevented can’t be measured in the same way a positive deliverable can, like the temperature of a room.

Metrics for goals are not just difficult to find; they are not very useful for security management. The reason for this is the indirect relationship between security activity and security goals. Intuitively most managers think that there is a direct link between what we do (which results or outputs) and what we want to achieve (the most important things: our goals). This belief is supported by real life experiences like making a sandwich. You buy the ingredients, go home, arrange them, and perhaps toast them and voilá: A warm sandwich ready to eat. The output sought (the sandwich) and the goal (eating a home made sandwich) match beautifully.

Unfortunately, there is no direct link every time. A good example can be research. There is not direct relationship between goals (discoveries) and the activity (experiments, publication). You can try hundreds of experiments and still not discover a cure for cancer. Same thing happens with security. The goals (trust, confidence, security) and the activity (controls, processes) are not directly linked.

When there is a direct link between activity and goal, like the temperature in a pot and the heat applied that pot, we know what decision to take if we want the temperature to drop: stop applying heat But, how will we make a network safer, adding (more accurate filtering), or summarizing (less complexity) filtering rules? We don’t know.  If a process produces dropped packets, more or less dropped packets won’t necessarily make the network more or less secure, just like a change in the firewall rules won’t necessarily make the network safer of otherwise.

The disconnect present in information security between goals and activity prevents goal metrics from being useful for management, as you can never tell if you are closer to your goals because of decisions recently taken on the security processes.

Goal metric examples:

• Instances of secret information disclosed per year. What can you do to prevent people with legitimate access to disclose that information.
• Use of system by unauthorized users per month. What can you do to prevent people from letting other users to use their accounts.
• Customers reports of misuse of personal data to the Data Protection Agency. Even if you are compliant, what can you do to prevent a customer to fill a report?
• Risk reduction per year of 10%. As risk depends on internal an external factors, what can you do to actually modify risk?
• Prevent 99% of incidents. How do you know how many incidents didn’t happen?

Actually useful security metrics
If metrics for goals are difficult to get, and are not very useful; what is a security manager to do? Measuring process outputs can be the answer. Measuring outputs is not only possible but very useful, as outputs contribute directly or indirectly to achieve security, trust and confidence. Using output metrics you can:

• Measure how changes in a process contribute to outputs;
• Detect significant anomalies in processes;
• Inform decisions to fix or improve the process.

There are seven basic types of process output metrics:

• Activity: The number of outputs produced in a time period;
• Scope: The proportion of the environment or system that is protected by the process. For example, AV could be installed in only 50% of user PCs;
• Update: The time since the last update or refresh of process outputs.
• Availability: The time since a process has performed as expected upon demand (uptime), the frequency and duration of interruptions, and the time interval between interruptions.
• Efficiency / Return on security investment (ROSI): Ratio of losses averted to the cost of the investment in the process. This metric measures the success of a process in comparison to the resources used.
• Efficacy / Benchmark: Ratio of outputs produced in comparison to the theoretical maximum. Measuring efficacy of a process implies the comparison against a baseline.
• Load: Ratio of available resources in actual use, like CPU load, repositories capacity, bandwidth, licenses and overtime hours per employee.

Examples of use of these metrics:

• Activity: Measuring the number of new user account created per week, a sudden drop could lead to detecting that the new administrator is lazy, or that users started sharing user accounts, so they are not requesting them any more.
• Scope: In an organization with a big number of third party connections, measuring the number of connections with third parties protected by a firewall could lead to a management decision not to create more unprotected connections.
• Update: Measuring the update level of the servers in a DMZ could lead to investigating the root cause if the level goes above certain level.
• Availability: Measuring the availability of a customer service portal could lead to rethinking the High Availability Architecture used.
• Efficiency / Return on security investment (ROSI): Measuring the cost per seat of the Single Sign On systems of two companies being merged could lead to choose one system over the other.
• Efficacy / Benchmark: Measuring backup speed of two different backup systems could lead to choose one over the other.
• Load: Measuring and projecting the minimum load of a firewall could lead to taking the decision to upgrade pre-emptively.

There is an important issue to tackle when using output metrics; what I call the Comfort Zone. When there are too many false positives, the metrics is quickly dismissed, as it is not possible to investigate every single warning. On the other hand, when the metric never triggers a warning, there is a feeling that the metric is not working or providing value. The Comfort Zone (not too many false positives, pseudo-periodic warnings) can be achieved using an old tool from Quality Management, the control chart. The are some rules used in Quality Management to tell a warning, a condition that should be investigated from a normal statistical variation (Western Electric, Donald J. Wheeler’s, Nelson rules), but for security management the best practice is adjusting the multiple of the standard deviation that will define the range of normal values for the metric until we achieve the Comfort Zone, pseudo-periodic warnings without too many false positives.

Using Security Management Metrics

There are six steps in the use of metrics: measurement, representation, interpretation, investigation and diagnosis.

Measurement: The measurement of the current value of the metric is periodic and normally refers to a window, for example: “9:00pm Sunday reading of the number of viruses cleaned in the week since the last reading” Measurements from different sources and different periods need to be normalized before integration in a single metric.

Interpretation: The meaning of a measured value is evaluated comparing the value of a measurement with a threshold, a comparable measurement, or a target. Normal values (those within thresholds) are estimated from historic or comparable data. The results of interpretation are:

• Anomaly: When the measurement is beyond acceptable thresholds.
• Success: When the measurement compares favourably with the target.
• Trend: General direction of successive measurements relative to the target.
• Benchmark: Relative position of the measurement or the trend with peers.

Incidents or poor performance take process metrics outside normal thresholds. Shewhart-Deming control charts are useful to indicate if the metric value is within the normal range, as values within the arithmetic mean plus/minus twice the standard deviation make more than 95.4% of the values of a normally distributed population. Fluctuations within the “normal” range would not normally be investigated.

Investigation: The investigation of abnormal measurements ideally ends with identification of the common cause, for example changes in the environment or results of management decisions, or a special cause (error, attack, accident) for the current value of the metric.

Representation: Proper visualization of the metric is key for reliable interpretation. Metrics representation will vary depending on the type of comparison and distribution of a resource. Bar charts, pie charts and line charts are most commonly used. Colours may help to highlight the meaning of a metric, such as the green-amber-red (equivalent to on-track, at risk and alert) traffic-light scale. Units, the period represented, and the period used to calculate the thresholds must always be given for the metric to be clearly understood. Rolling averages may be used to help identify trends.

Diagnosis: Managers should use the results of the previous steps to diagnose the situation, analyse alternatives and their consequences and make business decisions.

• Fault in Plan-Do-Check-Act cycle leading to repetitive failures in a process -> Fix the process.
• Weakness resulting from lack of transparency, partitioning, supervision, rotation or separation of responsibilities (TPSRSR) -> Fix the assignment of responsibilities .
• Technology failure to perform as expected -> Change / adapt technology.
• Inadequate resources -> Increase resources or adjust security targets.
• Security target too high -> Revise the security target if the effect on the business would be acceptable.
• Incompetence, dereliction of duty -> Take disciplinary action.
• Inadequate training -> Institute immediate and/or long-term training of personnel.
• Change in the environment -> Make improvements to adapt the process to the new conditions.
• Previous management decision -> Check if the results of the decision were sought or unintended.
• Error -> Fix the cause of the error.
• Attack -> Evaluate whether the protection against the attack can be improved.
• Accident -> Evaluate whether the protection against the accident can be improved.

What management practices become possible?
A side effect of an Information Security Management System (ISMS) lacking useful security metrics is that security management becomes centered in activities like Risk Assessment and Audit.  Risk Assessment considers assets, threats, vulnerabilities and impacts to get a picture of security and prioritize design and improvements while Audit checks the compliance of the actual information security management system with the documented management system with an externally defined management system or an external regulation. Risk Assessment and Audit are valuable, but there are more useful security management activities like monitor, test, design & improvement and optimization that become possible with output metrics. Theses activities can be described as follows:

• Monitor—Use metrics to watch processes outputs, detect abnormal conditions and assess the effect of changes in the process.
• Test—Check if inputs to the process produce the expected outputs.
• Improving -  Making changes in the process to make it more suitable for the purpose, or to reduce usage of resources.
• Planning – Organizing and forecasting the amount, assignment and milestones of tasks, resources, budget, deliverables and performance of a process.
• Assessment -  How well the process matches the organization’s needs and compliance goals expressed as security objectives. How changes in the environment or management decisions in a process change the quality, performance and use of resources of the process; Whether bottlenecks or single points of failure exist; Points of diminishing returns; Benchmarking of processes between process instances and other organizations. Trends in quality, performance and efficiency.
• Benefits realisation. Shows how achieving security objectives contributes to achieving business objectives, measures the value of the process for the organization, or justifies the use of resources.
  

While audits can be performed without metrics, monitoring, testing, planning,  improvement and benefits realisation are not feasible without them.

What needs to be done?
S.M.A.R.T security managers need metrics that actually help them performing management activities.

While it is not necessary to drop goal metrics altogether, the day to day focus of information security management should be on security monitoring, testing, design & improvement and optimization using output metrics, which are the ones which will show what are the effect of management decisions, if things are getting worse or better, if processes work as designed, and if there are changes out of our direct control that cause abnormal conditions in security processes. All these activities are perfectly feasible using outputs metrics and control charts.

The Stuff Information Systems are Made Of

There are quite a few Turing-complete computer models, among them:

• Busy Beaver

• Random Access Stored Program Machine
• Counter Machine
• Von Neumann Architecture

There are two reasons for using a information system model in Information Security. The first is that in order to understand the threats faced by information systems, we need to understand their components, what is their behaviour and how they relate to each other. The second is that security policies are too often tied too closely to actual hardware and software components and rendered obsolete by technological advances. Using a good model of an information system provides a necessary layer of abstraction that can help to make security policies mode durable.

A Turing machine has the following components:

• Tape: It stores information.

• Head: It read or writes information.

• Table: It specifies what will be the next action of the machine depending on the State and the current value under the Head.

• State register

Nowadays, most computers follow the Von Neumann Architecture. Operating systems that run on Von Neumann machines often follow the Unix convention, where the information system components are:

• File. This is equivalent to the “Tape”.

• Process. This is equivalent to the “State register”, the “Table”, and the “Head”.

As early information systems where not networked, the components needed for inter-system communication where not modelled. As a matter of fact, operating systems like Inferno try to keep the file/process model alive.

ISM3 represents information systems using a reduced set of components, but a set complex enough to illustrate how networked information system behave.

• Repositories: Any temporary or permanent storage of information, including RAM, databases, file systems, and any kind of portable media;

• Interfaces: Any input/output device, such as screens, printers and fax;

• Channels: Physical or logical pathways for the flow of messages, including buses, LAN networks, etc. A Network is a dynamic set of channels;

• Borders define the limits of the system.

• Services. Any value provider in an information system, including services provided by BIOS, operating systems and applications. A service can collaborate with other services or lower level services to complete a task that provides value, like accessing information from a repository;

• Sessions. A temporary relationship of trust between services. The establishment of this relationship can require the exchange of Credentials.

• Messages. Any meaningful information exchanged between two services or a user and an interface.

Examples:

Repository	Interface	Channel
Payroll Database	Web-based interface	HTTPS
Database Replica	System call	TCP
File system	Monitor, keyboard and mouse	Frame relay PVC
Hard drive	Connector	Cable

Service	Message	Session
Bank Account	Transfer from another account	Work session between user and application
SOAP API Interface	SOAP Call	Session between processes
Port	TCP Packet	TCP Transmission session
Ethernet Port	Ethernet Packet	Frame transmission session

Please note that:

• Repositories are linked with channels to other repositories and interfaces.

• Interfaces includes monitors, printers, any input/output device, and any connection between the borders of different information systems.

• A Channel can be a cable or any media that supports the transmission of an instance of communications (Message), like the air or vacuum for eletromagnetic waves.

• Services compute information from repositories and send Messages.

• Messages travel through channels between repositories. Using this construct strange expressions like “information in transit” can be avoided altogether. Note that a Message is different at every OSI level. At a high level, a mail or a transaction are messages. At lower levels, a message is an IP packet or even a eletrical signal.

• “Requests” are special Messages used by Services to change the state of information system components. Requests fall into one of the following categories:

Component	Initiate	Finalize	Freeze	Unfreeze	Query State	Change State
Session	login	logout	suspend	resume	read	write
Message	send	listen	retain	forward	read	write
Repository	create	delete	block	unblock	read	write
Interface	connect	disconnect	interrupt	continue	read	write
Channel	open	close	hold	release	read	write
Service	start	stop	pause	go	read	write

The Events Loggin Markup Language (ELML) is based on this information system model.

Some comments on the CEE MITRE effort: Their project is useful, necessary, and very ambitious, But IMHO they are missing an important opportunity to lay the foundation for a truly universal log format.  Their work discusses IDS, firewalls, honeypots, syslogs, and many other worthy topics; but it would be more useful in the long term to design a standard that can be applied at all OSI levels, not just at the networking and operating systems levels, but a standard that can be applied even to applications logging.

It seems that every log developer who deals with “Booking Rooms”, wants a “Booking Room” field in their log. If you check the mail list archive, there are discussions about taxonomies so powerful that they should be capable of representing any object/action that may possibly be logged. But a more simple approach based on an abstract model would suffice if the Semantics associated with common log events were separated within the base log record format as a user-defined description field.

The first question CEE MITRE should solve is ‘How can we represent everything an information system can do?’ and not ‘How can we represent everything we can use an  information system for?’ . An universal log format will certainly render a detailed answer to the first question. If the log format is extensible and provides adequate ‘holes’ for the expression of user semantics, a universal log format may also answer the second question.

Return On Security Investment

The information security industry recognizes both the necessity and the difficulty of carrying out a quantitative evaluation of ROSI, return on security investment.

The main reason for investing in security measures is to avoid the cost of accidents, errors and attacks. Direct costs of an incident may include lost revenues, damages and property loss, or direct economic loss. The total cost can be considered to be the direct cost plus the cost of restoring the system to its original state before the incident. Some incidents can cost information, fines, or even human lives.

The indirect cost of an incident may include damage to a company’s public image, loss of client and shareholder confidence, cash-flow problems, breaches of contract and other legal liabilities, failure to meet social and moral obligations, and other costs.

Measuring Return

What do we know intuitively about the risk and cost of security measures? First, the relationship between the factors that affect risk – such as window of opportunity, value of the asset and its value to the attacker, combined assets, number of incidents and their cost, etc. – is quite complex.  We also know that when measures are implemented to reduce risk, the ease of using and managing systems also decreases, generating an indirect cost of the security measures.

How do we go from this intuitive understanding to quantitative information? There is some accumulated knowledge of the relationship between investment in security measures and their results. First, there is the Mayfield paradox, according to which the cost of universal access to a system and absolutely restricted access is infinite, with more acceptable costs corresponding to the intermediate cases.

An empirical study was also done by the CERT at Carnegie Mellon University, which states that the greater the expenditure on security measures, the smaller the effect of the measures on security. This means that after a reasonable investment has been made in security measures, doubling security spending will not make the system twice as secure.

The study that is most easily found on the Internet on this subject cites the formulas created during the implementation of an intrusion detection system by a team from the University of Idaho.

R: losses.
E: prevented losses
T: total cost of security measures

(R-E)+T= ALE

R-ALE = ROSI, therefore ROSI = E-T

The problem with this formula is that E is merely an estimate, and even more so if the measure involved is an IDS, which simply collects information on intrusions, which means that there is no cause-effect relationship between detecting an intrusion and preventing an incident. Combining this type of estimate with basing it on mathematical formulas is like combining magic with physics.

What problems do we face in calculating return on investment of security measures? The most important is the lack of concrete data, followed closely by a series of commonly accepted suppositions and half-truths, such as that risk always decreases as investment increases, and that the return on the investment is positive for all levels of investment.

Nobody invests in security measures to make money; they invest in them because they have no choice. Return on investment demonstrates that investing in security is profitable, in order to select the best security measures with a given budget, and to determine whether the budget allocated to security is sufficient to fulfill the business objectives, but not to demonstrate that companies make money off of the investment.

In general, and also from the point of view of return on investment, there are two types of security measures: measures to reduce vulnerability and measures to reduce impact.

• Measures that reduce vulnerability barely reduce the impact when an incident does occur. These measures protect against a narrow range of threats. They are normally known as Preventive Measures. Some of these measures are firewalls, padlocks, and access control measures. One example of the narrowness of the protection range is the use of firewalls, which protect against access to unauthorized ports and addresses, but not against the spread of worms or spam.
• Measures that reduce impact to very little to minimize vulnerability if an incident does occur. These measures protect against a broad range of threats and are commonly known as Corrective Measures. Examples of these measures include RAID disks, backup copies, and redundant communication links. One example of the range of protection is the use of backups, which do not prevent incidents, but do protect against effective information losses in the case of all types of physical and logical failures.

The profitability of both types of measures is different, as the rest of the article will show.

Preventive or Vulnerability-Reduction Measures

A reduction in vulnerability translates into a reduction in the number of incidents. Security measures that reduce vulnerability are therefore profitable when they prevent incidents for a value that is higher than the total cost of the measure during that investment period.

The following formula can be used:

ROSI = CTprevented / TCP

CT  = Cost of Threat = Number of Incidents * Per Incident Cost.
TCP = Total Cost of Protection

When ROSI > 1, the security measure is profitable.

Several approximations can be used to calculate the prevented cost. One takes the prevented cost into account as the cost of the threat in a period of time before and after the implementation of the security measure.

CTprevented = ( CTbefore – CTafter)

Calculating the cost of the threat as the number of incidents multiplied by the cost of each incident is an alternative with respect to the traditional calculation of the incident probability multiplied by the incident cost, provided that the number of incidents in the investment period is more than 1. To calculate a probability mathematically, the number of favorable cases and the number of possible cases must be known. Organizations rarely have information on possible cases (but not “favorable” cases) of incidents. It is impossible to calculate the probability without this information. However, it is relatively simple to determine the number of incidents that occur within a period of time and their cost.

For a known probability to be predictive, it is also necessary to have a large enough number of cases, and conditions must also remain the same. Taking into account the complexity of the behavior of attackers and the organizations that use information systems, it would be foolish to assume that conditions will remain constant. Calculating the cost of a threat using probability information is therefore unreliable in real conditions.

One significant advantage of calculating the cost of a threat as the product of the number of incidents and their unit cost is that this combines the cost of the incidents, the probability, and the total assets (since the number of incidents partly depends on the quantity of the total assets) into a single formula. To make a profitability calculation like this, real information on the incidents and their cost is required, and gathering this information generates an indirect cost of an organization’s security management. If this information is not available, the cost of the threats will have to be estimated to calculate the ROSI, but the value of the calculation result will be low as the estimate can always be changed to generate any desired result.

The profitability of a vulnerability reduction measure depends on the environment. For example, in an environment in which many incidents occur, a security measure will be more profitable than in the case of another environment in which they do not occur. While using a personal firewall on a PC connected to the Internet twenty-four hours a day may be profitable, using one on a private network not connected to the Internet would not. Investing in a reinforced door would be profitable in many regions of Colombia, but in certain rural areas of Canada, this investment would be a waste of money.

Sample profitability calculation:

1. Two laptops out of a total of 50 are stolen in a year.
2. The replacement cost of a laptop is 1800 euros.
3. The following year, the company has 75 laptops.
4. The laptops are protected with 60€ locks.
5. The following year only one laptop is stolen.

ROSI = ( Rbefore – Rafter) / TCP

ROSI = ( ( 1800+Vi )*3 – (( 1800+Vi )*1+75*60) )/( 75*60 )

(The number of incidents is adjusted for the increase in the number of targets).

If a laptop was worth nothing (Vi=0), the security measure would not be profitable (ROSI < 1). In this example, the 60€ locks are profitable when a laptop costs more than 2700€, or when, based on historical information, the theft of 5 laptops can be expected for the  year in question.

Using this type of analysis, we could:

• Use locks only on laptops with valuable information.
• Calculate the maximum price of locks for all laptops (24€ when Iv=0).

Corrective or Impact-Reduction Measures

Since impact-reduction measures do not prevent incidents, the previous calculation cannot be applied. In the best case scenario, these measures are never used, while when there are two incidents which could result in the destruction of the protected assets, they are apparently worth twice the value of the assets. Now then, who would spend twice the value of an asset on security measures? Profitability of corrective measures cannot be measured. These measures are like insurance policies; they put a limit on the maximum loss suffered in the case of an incident.

What is important in the case of impact-reduction measures is the protection that you get for your money. The effectiveness of this protection can be measured, for example depending on the recovery time after an incident. Depending on their effectiveness, there are measures that range from backup copies (with some added cost) to fully redundant systems (which cost more than double).

One interesting alternative to calculating the ROSI of a specific security measure is to measure the ROSI of a set of measures – including detection, prevention, and impact reduction – that protect an asset. In this case, the total cost of protection (TCP) is calculated as the sum of the cost of all of the security measures, which the effort to obtain the information on the cost of the threats is practically identical.

Budget, cost, and selection of measures

The security budget should be at most equal to the annual loss expectancy (ALE) caused by attacks, errors, and accidents in information systems for a tax year. Otherwise, the measures are guaranteed not to be profitable. The graph below shows the expected losses as the area under the curve. To clarify the graph, it represents a company with enormous expected losses, of almost 25% of the value of the company. In the case of an actual company, legibility of the graph could be improved using logarithmic scales.

An evaluation of the cost of a security measure must take into account both the direct costs of the hardware, software, and implementation, as well as the indirect costs, which could include control of the measure by evaluating incidents, ethical hacking (attack simulation), audits, incident simulation, forensic analysis, and code audits.

Security measures are often chosen based on fear, uncertainty and doubt, or out of paranoia, to keep up with trends, or simply at random. However, the calculation of the profitability of security measures can help to select the best measures for a particular budget. Part of the budget must be allocated to the protection of critical assets using impact-reduction measures, and part to the protection of all of the assets using vulnerability-reduction measures and incident and intrusion detection measures.

Conclusions

The main conclusions that can be drawn from all of this are that:

• To guarantee maximum effectiveness of an investment, it is necessary, and possible if the supporting data is available, to calculate the return on the investment of vulnerability-reduction measures.
• In order to make real calculations, real information is needed regarding the cost of the incidents for a company or in comparable companies in the same sector.
• Both incidents and security measures have indirect and direct costs that have to be taken into account when calculating profitability.

How secret is a secret?

When a few know something and want to keep others from learning, that’s a secret. Everyone has secrets, some small, like eating a bar of chocolate when you are on a diet, some are personally important, like a embarrassing personal or professional mistake in the past. There are as many types of secrets as people and organizations that keep them, among them:

• Personal secrets and family secrets, normally related to the moral and taboos of the culture where they live.
• Business secrets, like financial information, strategy and trade secrets.
• Law enforcement secrets, like forensic or methods, investigation information and details about ongoing investigations.
• Crime secrets, like insider trading, organized crime and gangs.
• Political secrets, (Most nations have some form of Official Secrets Act and classify material according to the level of protection needed) like:

o Weapon designs and technology (nuclear, cryptographic, stealth).
o Military plans.
o Diplomatic negotiation positions.
o Intelligence information, sources and methods.
o International relations, secret treaties like:

• Molotov-Ribbentrop pact.
• Cuba crisis agreement.
• Dover treaty.
• Quadripartite agreement.
• Sykes-Picot agreement.

• Social secrets, like certain religions or secret societies as the masonry.
• Professional secrets, like health workers, social workers and journalists.
• Other, like video tape rental and sale records in the USA.

While we all have an intuitive way to distinguish small secrets from high secrets, there hasn’t been so far a way to measure it.

By using the following formula:

Secret = Log C*(Sum Tdk / Sum Tk) = Log C + Log ( Sum Tdk / Sum Tk )

Where C is the quantity of information, Tk is the time someone has known the secret, Tdk the time has had interest in knowing the secret.

If C=1, we can see some examples:

¿Who did “Famous for Nothing” went with for a dirty weekend last summer? If two people know since the 1st of August, 48 more since the 1st of September and 100.000 coach potatoes would like to know and they find out after five months, just before revelation the secrecy is:

S = Log ((100.000 * 5) / (2 * 5 + 48 * 4)) = 3,39

if only 8 people had found out in September, the secret would be S = 4,04

¿Who killed Kennedy? Let’s suppose two people know since 1963, and 300 million Americans would like to know. After 42 years:

S = Log( 300 million * 42 / 2 * 42 ) = 8,1

¿Who was Deep Throat? Just before it was found, 4 people knew, and 300 millions were interested. After 33 years:

S = Log( 300 million * 33 / 4 * 33 ) = 7,8

What if 2 more people had found out after 30 years?

S = Log( 300 million * 33 / (4 * 33 + 2 * 30) ) = 6,7

At a business, perhaps a secret is important for a few years, and all your competitors would be eager to know. If 10 people in the company know for a year, and 150 people from other companies would like to know:

S = Log( 150 * 1 / (10 * 1) ) = 1,17

If after two years the market is more competitive and more people (1500) is interested:

Secret = Log (1500 * 1 / (10 * 1) ) = 2,17

For the sake of example, I will use the following groups for measuring secrets:
• Family 10
• Social environment 100
• High school 300
• Competition 300
• Gang 50
• Police 100000
• Army 1000000
• Population 30000000
• Foreign Armies 15000000

Applying the formula, the following approximate values can server as examples of measured secrets:
• A social secret, like the confidence of a friend or an alibi of where you slept, 0,70
• Secret signs and telling signs from a gang, 0,95
• Hiding a mistake, like breaking something, 1,00
• Keeping the privacy of others like a confession, or the social situation of someone, 1,00
• Keeping the privacy of an average person like a list of video rentals of records of library use, 1,04
• Cheating on your wife/husband, 1,44
• Secrets from a Masonry, Scientology or Mirrahism, 1,48
• A regular trade secret, 2,18
• A mistake or wrongdoing of a politician 2,40
• Mafia, Yazuka or insider trading, 3,30
• Keeping the privacy of a politician like a list of video rentals of records of library use, 3,48
• Identity of a witnesses in a criminal case, 5,70
• A journalistic source, 5,78
• The Coca cola formula, 6
• Corruption, misuse of public funds 7,57
• Nuclear weapons, cryptography, 8,00
• Osama Bin Laden location 8,50

Mysteries, secrets known by no one, like those discovered by Champollion when deciphering Egyptian hieroglyphics have S=infinite.

Ignorance on the existence of the secret makes it less secret, as the interest on learning it is less, and the effort on keeping it secret is lower as well. Unfortunately it is very difficult to estimate the number of people interested in a secret, so the accuracy measuring secrecy won’t normally be very high. This way of measuring secrets can lead to some interesting exercises, like adding a factor to the formula for how intense is the interest for learning the secret or analyzing the diffusion of a secret in a group depending on the likelihood of every member of the group of revealing it.

Measuring secrecy can help to gain an understanding on how secret is the information we handle and the kind of efforts to make to keep it secret. To have a clear understanding on the reasons for keeping the secrecy and the influence of time, interest and group of people who know the secret, can give insights on how to manage secrets properly. Two conclusions can be easily drawn from the formula. Firstly, preventing others to know the existence of the secret makes it easier to keep it; Secondly keeping the group of people who know the secret as small as possible prevents leakages more effectively than any technical measure.

A clear understanding on the type of secrets in a organization, how secret they seem to be, the impact of their revelation and a measure of their secrecy is the first step to a cost effective and efficient classification and protection of secrets.

Note: This is an excerpt from an ISSA Journal article (April 2006). The formula was originally published in Kriptopolis.

Beyond authentication, authorization and accounting

A very common oversimplification of access control is: “authentication, authorization and accounting”. This narrow view obscures many subtle aspects of corporate practices for providing fine grained access to information resources.

Information systems represent users using user accounts or certificates and implement digital equivalents to guarded doors, records and signatures. While user accounts sometimes represent services or information systems instead of people, I will use the term “user” alone for simplicity.

Authentication is seen as a way to check for a users identity, but real implementations do not guarantee this. Every user account has an associated credential that is presented and validated by the system that controls the access to resources before the user of the user account can access those resources (this is called “login”). Upon login, we can’t assert the identity of the user, or even if the user account’s owner is the current user of the user account. What we do know is that someone who knows the user ID of a valid user account has presented to the system both the user ID and the associated credentials.

When we need to link user accounts and certificates to identifiable users, making users accountable for the use of the user account, we have to complement technical authentication with other controls.

The first control is checking the actual identity of the user before providing him with the set of user ID and credential. In countries were national ID cards exist, this control can be pretty straightforward. In other countries, widely different methods are used, normally checking if the user has information, papers and cards that only the legitimate user should have. I call this control User Registration.

The second control is preventing the user from sharing the credentials with other users. This can be achieved by using multiple credentials (biometric and password, for example)  Other enforcement controls are not as effective; for example one can warn the user of the consequences of sharing  the user ID and credential, and find ways to detect when this happens.

When protecting the anonymity of users is more important than making them accountable, we often need to guarantee that user accounts and certificates are not linked to identifiable users. This does not  mean that anonymous User Registration is trivial, as we often will want the users to fulfill certain criteria, what I call  Personality of the user. Common examples of personality test are: “Are you over 18?”, “Do you live in Europe?”, or “Are you human?”. Anonymous user accounts are not intended to check who the user is, but to check if the user is the same who used the user account the last time it logged in.

It is not rare to find online applications that don’t use use accounts at all, checking only for the personality of the user before providing access.

An additional aspect of authentication is session control, which limits:

• The number of login attempts with an invalid credential or user ID; what happens after the limit is reached; the time lapse between login attempts.
• The number of simultaneous sessions with the same user ID;
• The longest duration of a session.

So under the umbrella of “Authentication”, we find:

• Validation for the actual identity of identified users OR validation of the Personality of anonymous users (User Registration)
• Provision of user ID’s and credentials to users.
• Validation of the matching between user ID and credentials before grating any access to a resource.
• Limits on the usage of the system with a user ID, with or without a matching credential (Session Control)

“Authentication” therefore has three component proofs, which are mixed and matched as needed:

• Proof of identity.
• Proof of personality.
• Proof of ownership of the user account or digital certificate.

The traditional list of types of proof of ownership of the user account is:

• Something you know (passwords)
• Something you have (tokens and private keys)
• Something you are (biometrics)

This list is not complete, we can include:

• Something you like. What if we could test the taste for colors or music of a user, and reliably check it when the user wants to login?
• Something you can do. What if we could test the some ability of a user, like typing patterns, and reliably check it when the user wants to login?
• Something you think.  What if we could test the scale of values of a user, and reliably check it when the user wants to login? (I know, is far-fetched)

Authorization is seen as granting the use of resources to authorized users and deny it to unauthorized users. But, How does a user becomes authorized to use a resource?. A very simple scenario is when there is a Resource Owner, and Administrator, who manages the system, and a User. The user sends an “access rights request” to the resource owner, who can grant the request,  and ask the administrator to technically grant those rights on the resource to the user’s user ID. At a later time, the authorized user can use his user ID and credential to authenticate to the system, and use the resource using the rights he has been granted.

So under the umbrella of “Authorization” we find:

• Access Rigths Control
• Authorization

Under the the traditional Accounting concept of access control there are two distinct processes, Recording and Signing.

Recording registers accurately the results of access to resources, so these can be investigated and will and intent or responsibilities determined. The recording process will normally have to meet objectives for accuracy, including date and time. Recording normally registers;

• Interface ID and Location;
• User account or certificate ID;
• Signature;
• Type of Access Attempt (login, logout, change password, change configuration, connect/disconnect systems, repositories I/O interfaces, enabling/disabling admin access or logging, etc)
• Date and Time of Access attempt;
• Access attempt result;
• Resource accessed.

Signing records the will and intent about a “document” (or a mail, video, song, photo, etc) of the owner of the user account or certificate concerning a document, such as agreeing, witnessing or claiming authorship of documents like original works, votes, contracts and agreements. Digital signatures are a special kind of record.

A digital signature using public key encryption allows for authentication of documents, but what does a digital signature actually authenticate?. You can assert third parties authorship sometimes. For example “I know that this was written by Abel because it is signed with his private key, no one contests his authorship, and Abel doesn’t claim that his private key has been stolen”.

But what you can’t do is to assert your own authorship. Why? Because you can get a message from Abel, remove the digital signature and add your own, asserting that you are the author.

This is the reason why signing contracts can potentially become complicated. I could send you a signed contract, which you sign and send back for me to sign, which I could again sign to show I agree with your signing it, and send it to you so you can prove that I am committed to the contract, as the first signature only shows agreement to my own writing.

Summarizing, “Authentication, Authorization and Accounting” is an oversimplification for:

• User Registration.
• Provision of user ID’s and credentials to users.
• Authentication.
• Session Control
• Access Rights Control
• Authorization
• Recording
• Signing

I hope UPASAARS doesn’t become a popular acronym…it looks ugly, doesn’t it?

A final word on the equivalence between user accounts and digital certificates. A digital certificate serves the same purpose a user account does; providing and denying access to resources to people depending on what we know about them and how much we trust them.

For user accounts:

• User Registration, provision of user ID’s and credentials to users is performed in house.
• The user account has a user ID and a credential or set of credentials.
• Session Control is configured normally system by system.
• Access Rights are granted or denied depending on the users identity and personality.

For digital certificates:

• User Registration, provision of user ID’s and credentials to users is performed by a certification authority.
• The digital certificate has a Distinguished Name (equivalent to the user ID), and a credential (public key signed by a certification authority+private key) , which is a credential of the type “something you have”.
• Session Control is still configured system by system.
• Access Rights are granted or denied depending on how much we trust the guarantees the certification authority provides about the users identity and personality.

The advantage of digital certificates over user accounts is that they scale; we can trust users that haven’t gone through our own User Registration,  the disadvantage is that we can trust them only as much as we trust their Certification Authority regarding their User Registration quality in terms of how do they check their users identity and personality.

This is what OpenID ultimately tries to accomplish, without relying solely on digital certificates.

Management level threats

A threat causes harm sometimes helped by a weakness, sometimes impeded by a countermeasure.

A threat has an agent, a mechanism and consequences for an information system or repository.

Using agent and consequences for classification, threats can be classed as Errors (unintentional human action), Attacks (intentional human action) and Accidents (&Disasters) (non-human action).

The consequences of an Attack, Error or Accident can be:

1. Failure to destroy of repositories or messages
2. Destruction or Loss of repositories or messages
3. Theft of repositories or messages
4. Interruption of repositories or messages
5. Corruption of repositories or messages
6. Outdated repositories or messages
7. Unauthorized access, Disclosure of repositories or messages
8. Improper use of authorized access of repositories or messages
9. Improper recording of access to services, channels or interfaces
10. Failure to stop services, channels or interfaces
11. Destruction or Loss of services, channels or interfaces
12. Eavesdropping of services, channels or interfaces
13. Underperformance or Interruption of services, channels or interfaces
14. Corruption of services, channels or interfaces
15. Unauthorized use of services, channels or interfaces
16. Improper use of authorized access of services, channels or interfaces
17. Improper recording of use of services, channels or interfaces
18. Aging of services, channels or interfaces

While some will argue that the mechanism of the threat is important, I don’t think it is always necessary. There are hundreds of different and subtle ways to attack a system. Is it necessary to analyze every single way, or is it better to design and protect the systems in a way that makes it resilient to any threat?

For example a good backup process can protect any system from several of these threats…

Business Modelling for Security

One of the first steps for a new ISMS implementation project is finding out what would be the ISMS best suited to the company goals, that the organization can afford. As an incident is an attack, accident or error that prevents a business objective to be met, it is necessary to find out what those business objectives are. Generally speaking the goals of any company are:

• Achieving a vision and mission;
• Continuing to exist;
• Maintaining and growing revenue;
• Attract, maintain and fostering talent;
• Maintaining and growing brand and reputation;
• Complying with internal ethics and social responsibility goals;
• Complying with regulations and contracts;

The more specific we can get, the better design of the ISMS will result. It is possible to add granularity to the analysis of business objectives, using the the following list business functions:

1. Governance: Definition of the organization’s goals, steering of the organization by rules, instruction and challenge rules and instructions.
2. Research. Creation of new knowledge in every area of interest to the organization.
3. Advertising. Promotion of the organization’s services and products to potential customers, suppliers and investors.
4. Business Intelligence. Maintenance and delivery of knowledge.
5. Human Resources. Finding, selecting and procuring, promoting and releasing personnel.
6. Information Technology. Finding, filtering and procuring information and communication systems.
7. Legal. Claiming legally binding obligations from third parties and fulfilling the organization’s own.
8. Relationships. Creating and maintaining trust, association and familiarity with customers, suppliers, and investors.
9. Administration. Management of paperwork associated with all business functions..
10. Financing / Accounting. Finding, selecting and procuring financial instruments like e.g. money, bonds, etc.
11. Infrastructure. Management of real estate, air conditioning, heating, water supply, energy supply, furniture, food supply, waste , recycling , physical access control, etc
12. Logistics. Delivery of physical products or services.
13. Maintenance. Preventing and repairing faults and the general dilapidation of infrastructure, tools, etc
14. Procurement. Finding, comparing, choosing, selecting and procuring information, tools, supplies, assets and professional services.
15. Production. Production of products and services.
16. Sales: Sale of products or services.

A top down approach (What is the business about?) can deliver superior results than the bottom up approach (How important to the business is this particular IT system? And this one? And this one?)

ISM3-RA uses this simplest business model, there many other, sometimes more complicated ways to model a business.

Audit Standards vs Management Standards

Alex Hutton makes an interesting point in his post “There’s nothing wrong with the PCI DSS“. Audit standards are made by and for auditors, while management standards should be made for managers, who do many things other than auditing. An audit standard helps you to understand and show if you are doing things the way a third party thinks you should. A management standard helps you figure out how to do the things you business needs you to do. Isn’t it interesting that the most popular ISO 27001 course is “ISO 27001 Lead Auditor” and not “ISO 27001 Manager”?

I have to disagree with Alex when he says that “ISM^3 can’t tell you where or how to go get many of the measurements we need”. Metrics in ISM3 are process metrics that measure:

• Activity: The number of outputs produced, their mean age, the mean time between outputs submissions, mean time to produce an output, following input, and worst case time to produce an output, following input.
• Scope: The proportion of the environment or system that is protected by the process and the percentage of the scope sampled.
• Unavailability: The time since a process has performed as expected upon demand (uptime), the frequency and duration of interruptions.
• Effectiveness: Number of inputs, mean time between inputs, and percentage of inputs that produce an output.
• Efficiency: Ratio between the number of outputs submitted and the available resources for this process in actual use.
• Load: Percentage of resources in actual use.
• Quality: Accuracy, precision or other measurements of fitness for purpose of the output, when applicable.

I think these are exactly the measurements managers need for Planning, Benefits Realisation, Monitoring, Testing, Assessment and Improvement of security processes.

ISM3 v2.3 published

The main novelties are:

• Capability is not subjective any more. It depends on what types of metrics are used to manage every process. ISM3 is the first method that defines capability this way.
• Metric types are now 7 instead of 4. Activity, Unavailability, Scope, Load, Quality, Efficacy and Efficiency.
• GP-1 Document Management is updated to GP-1 Knowledge Management.
• OSP-23 Events Detection and Analysis is updated to OSP-23 Internal Events Detection and Analysis.
• TSP-6 Define environments and lifecycles is updated to TSP-6 Security Architecture
• New process OSP-28 External Events Detection and Analysis takes care of reputation, copyright violations and phishing.
• New process TSP-14 Information Operations includes intelligence and misinformation.
• Maturity levels have been renamed as follows: Basic Level, SME Level, eCommerce Level, Enterprise Level and Military Level.
• Enhanced metric management guidance (Measurement-Interpretation-Investigation-Representation-Diagnosis)

Today, the ISM3 Consortium published the print version of Information Security Management Maturity Model (ISM3) v2.3. The method has been updated with security management metrics proven in the field, and a new approach that defines security maturity objectively as a direct result of the metrics used to manage information security processes.

ISM3 focuses on “Achievable Security” rather than “Absolute Security”. Achievable security is a trade-off between absolute security and business requirements. The traditional view that “Information Security should prevent all attacks” is not realistic for most organizations. ISM3 achieves its balance by mapping an organization’s business objectives (such as product delivery and profitability) directly against security objectives (such as ensuring data access only to authorized users).

ISM3 builds on successful principles from the field of quality management (Six Sigma, ISO9001), and applies these ideas to the field of information security, providing an opportunity for organizations of all types and sizes to enhance their ISM systems and align them with their business needs. Implementations of ISM3 are compatible with ISO27001, which establishes control objectives for each process.  Implementations use management responsibilities framework similar to the IT Governance Institute’s CobIT framework model, which describes best practices in the parent field of IT service management. ITIL users can use ISM3 process orientation to seamlessly strengthen ITIL security process. Using ISM3 style metrics, objectives, and targets it is possible to create measurable Service Level Agreements for outsourced security processes.

The significant features of ISM3 are:

• Metrics for Information Security – “What you can’t measure, you can’t manage, and what you can’t manage, you can’t improve” – ISM3 v2.3 is probably the first information security standard to make information security a measurable process by using metrics for every process. This allows continuous improvement, as the standard defines criteria to measure efficiency and performance.
• Capability Levels – ISM3 is the first standard that defines capability in terms of metrics, a leap that makes ISM3 orientation to continuous improvement unique.
• Maturity Levels – ISM3 comes in five different sizes, or maturity levels. This makes it suitable for a wide range of organizations, from the very large to the very small. Each maturity level is tailored to the security objectives of the target organization.
• Process Based – ISM3 v2.3 is process based, which makes it specially suited to organizations familiar with ISO9001 and those that use ITIL as the IT management model. It also works well for outsourced services as it provides a common language for collaboration between information security clients and providers.
• Adopts best practices – implementation of ISM3 is facilitated by its extensive cross-references to other established standards. The IT governance model reflects best practices by clearly distributing responsibility for information security processes between strategic, tactical and operational levels of management.
• Accreditation – ISM systems based on ISM3 can be certified under ISO9001 or ISO27001 systems, and ISM3 can be used as a tool to implement an ISO27001 ISM system. This should increase its attractiveness to organizations that already hold quality certification or have experience with ISO9001.

Purchase the method from lulu.com.
Learn more about ISM3
Learn more about the Consortium

The dangers of narrow scopes of applicability

Accreditation of an ISMS can give you several choices. One choice is your Risk Assessment method, another is your Scope, expressed in the Statement Of Applicability, and you can choose to leave some controls out as well, if you can explain why they don’t apply to you.

Choice is generally speaking good. But for accreditation, this brings a reputation issue. The reputation of a certificate holder is as good as the market perception of the performance of the worst of all certificate holders.  Education certifications and diplomas, for example, carry more reputation the LESS choice you have in your studies, not the more choice you have. Doctors can’t choose not to take Anatomy, but Arts can study nearly anything (depending on what country are you based)

The existence and significance of the Statement Of Applicability is well beyond anyone how is not a specialist. This means it is possible to choose a very narrow SOA, totally unrelated to your critical systems for the sake of getting accredited, regardless of your real information security posture. This is bad for certificate holders that choose real SOAs, as their competition can get the same reputation for a far smaller investment.  Another side effect of choice (SOA et al) is that a big financial company with several sites get as easily certified as a small technology company with only one site.  I think that is BAD, as the effort, resources and quality of implementation can be quite different. If I ran a big company I wouldn’t specially like to spend a lot time, effort and resources to get a certificate that just anyone can have, doing far less.  Another side effect of wild choice  is that a big financial company with several sites get as easily certified as a small technology company with only one site. Again, I think this is BAD, as simple technology infrastructure should be simpler to secure than a complex one.