Teradata Architecture 6dy5n

Teradata Architecture

AUTHOR:

JAYAKRISHNAN.V

E-MAIL : [email protected]

Teradata corporation Teradata corporation is a vendor specializing in data warehousing and analytic applications . • Its products are commonly used by companies to manage data warehouses for analytical and business intelligence purpose . • Teradata was formerly a division of NCR Corporation

Features • Teradata is Relational Database Management System (RDBMS) – Used for Data warehousing – Executes on Unix ,Windows NT or Windows 2000 operating systems Win NT

Win 98

Teradata DATABASE IBM Mainframe

UNIX

• Compliant with ANSI standards • Capable of running in single node or in multiple nodes • Unlimited, Proven Scalability - amount of data and number of s; allows for an enterprise wide model of the data. • Unlimited Parallelism - Parallel access, sorts, and aggregations. • Mature Optimizer - Handles complex queries, up to 64 s per query, ad-hoc processing.

• Models the Business - 3NF, robust view processing, & provides star schema capabilities • Low TCO (Total Cost of Ownership) - ease of setup, maintenance, & istration; no re-orgs, lowest disk to data ratio, and robust expansion utility (recon fig). • High Availability - no single point of failure. • Parallel Load utilities - robust, parallel, and scalable load utilities such as Fast Load, MultiLoad, and TPump.

Teradata Components & Architecture Client Req. Network attached system

Client Req Channel attached system

CLI MTDP

CLI TDP

MOSI

Channel Adapter Bus Adapter

Ethernet Adapter

Gateway S/W Parser Engine

Parser Engine

Message ing Layer AMP

AMP

AMP

AMP

Vdisk

Vdisk

Vdisk

Vdisk

Teradata RDBMS Components By 2 ways we can connect to Teradata RDBMS • 1) Network attached system - through windows, Linux, UNIX • 2) Channel attached system- through Mainframe

Network attached system • Components a) CLI- call level interface (lowest level interface) > It creates sessions, allocates requests and response and also fetches responses b) MTDP – Micro Teradata Director Program > Controls the session related issues

d) MOSI – Micro Operating s/m interface -> It provides OS independent interface • For connecting to Teradata RDBMS network attached system needs Ethernet adapter

Channel attached system • CLI is having the same function as in n/w attached s/m • TDP – Teradata Director Program • Manages session traffic b/n CLI n Teradata database, session initiation and termination, logging, verification, session balancing etc • For connecting to Teradata RDBMS channel attached system needs host channel adapter and bus adapter in Teradata RDBMS

Teradata RDBMS components • 4 main components • 1) PE – Parser engine • 2) MPL – Message ing layer • 3) AMP – Access module processor • 4) PDE –Parallel database extension

The Parsing Engine Answer Set Response

SQL Request

Parser Parsing Engine

Optimizer Dispatcher

Message ing Layer

AMP

AMP

AMP

AMP

• The Parsing Engine is responsible for: – Managing individual sessions (up to 120) – Dispatching the optimized plan to the AMPs – Input conversion (EBCDIC / ASCII) - if necessary – Sending the answer set response back to the requesting client

• 3 main components for PE a)Parser and Resolver • Checks for syntax errors • Checks the access permission for requested db object • Check the existence of requested object and also returns error message if appropriate access is not there. b)Optimizer • It prepares access plan and Explain plan which shows how my query is solved efficiently means it restructure the query in such a way that it will run more efficiently c)Generator • Takes the explain plan created by optimizer and converts it into db understandable language called AMP steps

2) MPL – Message ing layer -> Handles the internal communication of Teradata dbms -> AMP steps are distributed to corresponding AMP based on the Hashing algorithm over the Message ing Layer (BYNET)

3) AMP – Access module processor -> Virtual processors running under a multitasking environment -> BYNET interface -> Manage database -> Interface to disk subsystem

The Access Module Processor (AMP) Answer Set Response

SQL Request

Parsing Engine

Message ing Layer (PDE and BYNET)

AMP

AMP

AMP

AMP

AMPs store and retrieve rows to and from disk

• The AMPs are responsible for: - Finding the rows requested - Lock management - Sorting rows - Aggregating columns - processing - Output conversion and formatting - Creating answer set for client - Disk space management - ing - Special utility protocols - Recovery processing

` 4) PDE –Parallel database extension This component is an interface layer on the top of operating system. Its functions include executing vprocs (virtual processors), providing a parallel environment, scheduling sessions, debugging, etc.

Teradata Storage Architecture Records From Client (in random sequence) 2

32

67

12

90

6

54

75

18

25

80

41

Teradata

Parsing Engine(s)

Message ing Layer

AMP 1

2 18

AMP 2

12

54 41

AMP 3

AMP 4

80 90

75

32

67

25 6

• The Parsing Engine dispatches request to insert a row. • The Message ing Layer insures that a row gets to the appropriate AMP (Access Module Processor). • The AMP stores the row on its associated (logical) disk. • An AMP manages a logical disk which is mapped to multiple physical disks in a disk array.

Teradata Retrieval Architecture Rows retrieved from table 2

32

67

12

90

6

54

75

18

25

80

41

Teradata Parsing Engine(s)

Message ing Layer

AMP 1

2 18

AMP 2

AMP 3

12

80

54 41

90

75

AMP 4

32

67

25 6

• The Parsing Engine dispatches a request to retrieve one or more rows. • The Message ing Layer insures that the appropriate AMP(s) are activated. • The AMP(s) locate and retrieve desired row(s) in parallel access. • Message ing Layer returns to retrieved rows to PE. • The PE returns row's to requesting client application.

Teradata Parallelism • Each PE can handle up to 120 sessions in parallel. • Each Session can handle multiple REQUESTS. • The Message ing Layer can handle all message activity in parallel. • Each AMP can perform up to 80 tasks in parallel. • All AMPs can work together in parallel to service any request. • Each AMP can work on several requests in parallel.

Multiple Tables on Multiple AMPs EMPLOYEE Table

DEPARTMENT Table

JOB Table

• Some rows from each table may be found on each AMP. Parsing Engine

• Each AMP may have rows from all tables.

Message ing Layer

AMP 1

AMP 2

AMP 3

AMP 4

EMPLOYEE ROWS EMPLOYEE ROWS EMPLOYEE ROWS EMPLOYEE ROWS DEPARTMENT ROWS DEPARTMENT ROWS DEPARTMENT ROWS DEPARTMENT ROWS JOB ROWS JOB ROWS JOB ROWS JOB ROWS

• Ideally, each AMP will hold roughly the same amount of data.

Multi-Node MPP System BYNET

SMP

DAC

DAC

SMP

DAC

DAC

SMP

DAC

DAC

SMP

DAC

DAC

• Multiple nodes may be configured to provide a Massively Parallel Processing (MPP) system. • A physical message ing layer called the BYNET is needed to interconnect multiple nodes. • Teradata is a linearly expandable RDBMS - as your database grows, additional nodes may be added.

BYNET (for MPP)

BYNET 0

SMP

SMP

SMP

• BYNET Features:

BYNET 1

SMP

SMP

SMP

SMP

SMP

The BYNET is a dual redundant, bi-directional interconnect network. All SMPs are connected to both BYNETs.

– Enables multiple SMP nodes to communicate. – Automatic load balancing of message traffic. – Automatic reconfiguration after fault detection. – Fully operational dual BYNETs provide fault tolerance. – Scalable bandwidth as nodes are added. – Even though there are two physical BYNETs to provide redundancy and bandwidth, Teradata and T/IP software only see a single network.

Teradata file system • Is a layer between Teradata RDBMS and PDE • Also provides set of service calls that allows Teradata RDBMS to store and retrieve data efficiently

Disk Arrays

• • • • •

A disk array contains drive groups Drive groups contain set of drivers LUN (Logical units) contains a portion of every drive pdisk is a slice of LUN Group of pdisks assigned to an AMP is called vdisk

Data Distribution

• According to the primary index selected, the rows get distributed randomly among all the AMP's.

• More the unique the primary index, the more even the distribution will be.

THANKS