This summarizes RUSH's escalation matrix and business continuity framework.
1. Escalation Matrix
1.1 Table 1: Contact Information and Resolution Time for Incidents and Requests. For incidents that concern third-party enablers of the Service such as but not limited to Logistics Providers and Payment Gateways, these incidents are not covered by the prescribed resolution time indicated on Table 1.
L1 Support shall be provided by the Merchant while RTI shall provide L2 and L3 Support through the following channels and operating schedule:
- Email: support@rush.ph
- Availability: 8:00AM to 5:00PM, Mondays to Fridays
- Helpdesk Link: https://help.rush.ph/knowledge
- Availability: Anytime
Complete information regarding Resolution Time based on severity can be found in this link: https://help.rush.ph/knowledge/resolution-time-based-on-severity
1.2 Table 2: Escalation Matrix for Incidents and Requests. In the event that RTI fails to respond within the given response time based on Table 1, the following escalations shall apply:
Escalation Level | Designation | Email/URL | How/When to Escalate |
1 | Merchant Support | Solution Support support@rush.ph |
Once the issue is identified to be under the scope of RUSH |
2 |
Merchant Support Manager |
Julia Dane Ubaldo julia@rush.ph |
If escalation 1 is unresponsive given the time frame of the resolution SLA table. |
3 | Chief Operations Officer | Jeff Alejandrino jeffrey@rush.ph |
If escalations 1 and 2 are unresponsive given the time frame of the resolution SLA table. |
4 | Chief Executive Officer | Stephanie Kubota steph@rush.ph |
If escalations 1, 2, and 3 are unresponsive given the time frame of the resolution SLA table. |
2. Business Continuity Framework
Rush Technologies, Inc. Business Continuity Plan
Introduction
This Business Continuity Plan (BCP) was developed to enable Rush Technologies, Inc. to maintain and/or resume critical operations and services to clients at functional levels following major disruptions and recover to full capabilities within prescribed timeframes.
RTI reserves the right at any time, and from time to time, to modify this BCP as the RUSH Software may reasonably require according to RTI sole discretion with or without notice (unless otherwise required by applicable law). Said revisions or modifications shall be posted in the RUSH website, and once published therein, shall become binding on Merchant. It shall be Merchant’s obligation to be informed thereof by
accessing, from time to time, such website where the latest version of this BCP may be found.
Policy Statement
This document provides information and guidance for maintaining critical business operations and protecting staff health and safety, as well as information needed during a recovery.
This document is primarily addressed to the senior and middle-management teams to enable them to prepare for ‘disasters and crises’ that disrupt normal business operations and/or threaten the access and availability of resources -- personnel, records, information, and physical facilities.
Key Principles
This document aims to:
- Ensure that all Rush Technologies, Inc. employees understand their responsibilities in relation to the BCP
- Provide contingency arrangements are appropriate, agile and cost-effective
- Align all decisions following disruptions in all departments
- Enable disaster recovery capabilities as applicable to key customers, vendors and other stakeholders
Crisis Management process
The President/CEO will notify his/her Mancom team of an emergency situation and the company’s immediate plans and they, in turn, will notify their top-tier managers. The top-tier managers will serve as the focal points for their departments and will call the people they have been assigned to call. The call tree continues until all affected personnel have been contacted.
Call Tree Flow and Guidelines
- No one person should call more than 5 names. By keeping each call to less than 3 minutes, 100+ employees can be contacted in less than 51 minutes.
- If person called is available, relay the information
- disaster status
- action to be taken
- If a person called is not available, call the next in the list. If a person remains uncontactable for 20 minutes,
- call the alternate coordinator or POC or
- If there is no alternate POC, call the next person that this person is assigned to notify
- Report list of uncontactable persons to your coordinator/POC
Key Business Process and BCP Strategy
Key business processes and the agreed BCP strategy for each are listed below. The strategy chosen for 24/7 operation is to work alternatively from home (teleworking) or a temporary site that will be provided by management if office building facility recovery will take longer. Monitoring tools, knowledge base and relevant documents are hosted and secured in the cloud.
Systems and IT infrastructure are deployed in a multi-site cloud providing high availability. In case one of the availability zones becomes inaccessible, the affected server can be easily restored to another availability zone using snapshots.
Key Business Process | BCP Strategy |
Rush Technologies, Inc. Operations |
|
Workstations/Workplace | Laptop Computer / Work from home |
Internet connection | Pocket Wifi / Mobile Phone / VPN |
Storage | Google Drive / AWS S3 |
Telephone | Mobile Phone |
Systems & Infrastructure | |
Servers | AWS Multi-zone deployment / Snapshots / S3 / AWS Terraform |
Databases | AWS Multi-zone deployment / Replication / Snapshots / Backups |
Office Facilities | Alternative Site - 917Ventures in The W Fifth Avenue, 5th Ave, BGC, Taguig, Metro Manila, Philippines Secondary - The Globe Tower in 32nd St, BGC, Taguig, Metro Manila, Philippines Teleworking (Work from Home) |
Communications | |
Internal Meetings | Google Meet / Zoom |
External Meetings | Google Meet / Zoom / MS Teams / AWS Chime |
Event Categories
Potential disasters have been assessed as follows:
Potential Disaster | Probability Rating | Impact Rating |
Flood | 1 | 2 |
Fire | 3 | 1 |
Earthquake | 5 | 1 |
Civil Unrest | 2 | 4 |
Typhoon | 2 | 2 |
Infectious Disease Outbreak | 5 | 2 |
Data/Hacking | 1 | 2 |
Probability: 1 = very high, 5 = very log Impact: 1 = total destruction, 5 = minor annoyance
Business Continuity Points of Contact
- People and Governance Director
- Merchant Experience Director
- Solutions Head
- Infrastructure Head
- Testing Center of Excellence (TCOE) Head
Emergency Response
Plan Triggering Events
Key trigger issues at main office that would lead to activation of the Emergency Response are:
- Total loss of all communications due to fire, earthquake and typhoon
- Total loss of power due to fire, earthquake and typhoon
- Flooding of the premises / Inaccessible premise
- Loss of the building due to fire and earthquake
- Infectious Disease Outbreak
- Emergency Response Guideline
Emergency Response Guideline
Alert Level | Response |
Severity 3 | Business As Usual Management:
|
Severity 2 | Discretionary Management:
Employee:
|
Severity 1 | Activate BCP Management:
Employee:
|
When an incident occurs the Emergency Response Team (ERT) must be activated. The ERT will then decide the extent to which the Disaster Recovery Plan must be invoked. All employees must be issued a Quick Reference card containing ERT contact details to be used in the event of a disaster. Responsibilities of the ERT are to:
- Respond immediately to a potential disaster and call emergency services;
- Assess the extent of the disaster and its impact on the business, cloud asset, data center, etc.;
- Decide which elements of the DR Plan should be activated;
- Establish and manage disaster recovery team to maintain vital services and return to normal operation;
- Ensure employees are notified and allocate responsibilities and activities as required.
Evacuation and Assembly Points
Where the premises need to be evacuated due to Earthquake or Fire, the DRP invocation plan identifies evacuation assembly point:
Primary – Nearest mall (Highstreet in BGC; Pioneer Center in Globe Pioneer, etc)
Alternate Recovery Facilities
If necessary, the hot site at 917Ventures in The W Fifth Avenue, 5th Ave, BGC, Taguig, Metro Manila, Philippines will be activated and notification will be given with managers. Hot site staffing will consist of members of the disaster recovery team only for the first 24 hours, with other staff members joining at the hot site as necessary.
Secondary hot site will be The Globe Tower in 32nd St, BGC, Taguig, Metro Manila, Philippines.
Activation of Disaster Recovery Team
The Disaster Recovery Team (DRT) is responsible for activating the Disaster Recovery Plan for potential disasters identified in this document as well as in the event of any other occurrence that affects the company’s capability to perform normally. These include but are not limited to office building structures, information systems or IT infrastructures. The DRT will then decide the extent to which the DRP must be invoked.
Responsibilities of the DRT are to:
- Assess the extent of the disaster and its impact on the business, cloud asset, data center, etc.;
- Decide which elements of the DR Plan should be activated;
- Ensure employees are notified and allocate responsibilities and activities as required.
- Establish facilities for an emergency level of service within 1.0 business hours;
- Restore key services within 4.0 business hours of the incident;
- Recover to business as usual within 8.0 to 24.0 hours after the incident;
- Coordinate activities with the disaster recovery team, first responders, etc.
The DR plan will rely principally on key members of management and staff who will provide the technical and management skills necessary to achieve a smooth technology and business recovery. Suppliers and service providers should continue to support recovery of business operations as the company returns to normal operating mode.
Disaster Recovery Plan
Crisis Communications and Flowchart
The Rush Technologies, Inc’s President/CEO will notify his/her Mancom team of an emergency situation and the company’s immediate plans and they, in turn, will notify their top-tier managers. The top-tier managers will serve as the focal points for their departments and will call the people they have been assigned to call. The call tree continues until all affected personnel have been contacted.
Business Continuity Documentation Maintenance
Document Review
To ensure that BCP is implemented and operated in accordance with the company’s policies and procedures approach to managing information security and its implementation (i.e. policies, processes and procedures for information security) shall be reviewed at planned intervals (at least once a year) or when significant changes occur.