Файловый менеджер - Редактировать - /home/digitalm/venv/lib/python3.7/site-packages/pandas/io/sas/__pycache__/sas_xport.cpython-37.pyc
Назад
B �5�g�6 � @ s. d Z ddlmZ ddlmZ ddlZddlmZmZ ddlZddl Z ddl m Z ddl ZddlmZ ddlmZ d Zd Zd Zd Zd ddddddddddddddd gZd Zd Zd Zd Zd!e� d"e� d"e� d"e� d#� Z d$e� d"e� d%�Z d&Z e ed'�d(d)�Z e d*�d+d,�Z!d-d.� Z"d/d0� Z#G d1d2� d2eej$�Z%dS )3z� Read a SAS XPort format file into a Pandas DataFrame. Based on code from Jack Cushman (github.com/jcushman/xport). The file format is defined here: https://support.sas.com/techsup/technote/ts140.pdf � )�abc)�datetimeN)�IO�cast)�Appender)� get_handle)� ReaderBasezPHEADER RECORD*******LIBRARY HEADER RECORD!!!!!!!000000000000000000000000000000 zKHEADER RECORD*******MEMBER HEADER RECORD!!!!!!!000000000000000001600000000zPHEADER RECORD*******DSCRPTR HEADER RECORD!!!!!!!000000000000000000000000000000 zPHEADER RECORD*******OBS HEADER RECORD!!!!!!!000000000000000000000000000000 �ntypeZnhfun� field_lengthZnvar0�name�labelZnformZnflZ num_decimalsZnfjZnfillZniformZniflZnifdZnpos�_z�Parameters ---------- filepath_or_buffer : str or file-like object Path to SAS file or object implementing binary read method.z�index : identifier of index column Identifier of column that should be used as index of the DataFrame. encoding : str Encoding for text data. chunksize : int Read file `chunksize` lines at a time, returns iterator.zBformat : str File format, only `xport` is currently supported.z\iterator : bool, default False Return XportReader object for reading file incrementally.z#Read a SAS file into a DataFrame. � a Returns ------- DataFrame or XportReader Examples -------- Read a SAS Xport file: >>> df = pd.read_sas('filename.XPT') Read a Xport file in 10,000 line chunks: >>> itr = pd.read_sas('filename.XPT', chunksize=10000) >>> for chunk in itr: >>> do_something(chunk) z$Class for reading SAS Xport files. z� Attributes ---------- member_info : list Contains information about the file fields : list Contains information about the variables in the file z�Read observations from SAS Xport file, returning as data frame. Parameters ---------- nrows : int Number of rows to read from data file; if None, read whole file. Returns ------- A DataFrame. )�datestr�returnc C s( y t �| d�S tk r" tjS X dS )z1Given a date in xport format, return Python date.z%d%b%y:%H:%M:%SN)r �strptime� ValueError�pdZNaT)r � r �M/home/digitalm-up/venv/lib/python3.7/site-packages/pandas/io/sas/sas_xport.py� _parse_date� s r )�sc C sD i }d}x0|D ](\}}| ||| � � � ||< ||7 }qW |d= |S )a Parameters ---------- s: str Fixed-length string to split parts: list of (name, length) pairs Used to break up string, name '_' will be filtered from output. Returns ------- Dict of name:contents of string at given location. r r )�strip)r �parts�out�startr �lengthr r r � _split_line� s r c C sT |dkrPt �t| �t �d��}t �d|� dd| � ��}|j|d�}| |d< |S | S )N� ZS8�Sz,S)�dtype�f0)�np�zeros�lenr �view)�vec�nbytes�vec1r Zvec2r r r �_handle_truncated_float_vec� s r) c C s t �d�}| j|d�}|d }|d }|d@ }t jt| �t jd�}d|t �|d@ �< d|t �|d @ �< d |t �|d @ �< ||L }||? |d @ d d | > B }|dM }||d? d@ d d> | d d> |d@ B O }t jt|�fdd�}||d< ||d< |jdd�}|�d�}|S )zf Parse a vector of float values representing IBM 8 byte floats into native 8 byte floats. z>u4,>u4)r r! �f1i��� � i � i @ � i � � � l �� � � �A i� � l z>f8Zf8) r" r r% r# r$ Zuint8�where�emptyZastype) r& r r( Zxport1Zxport2Zieee1�shiftZieee2Zieeer r r �_parse_float_vec� s( r7 c @ sl e Zd ZeZddd�Zdd� Zdd� Zd d � Zd d � Z e d �dd�Z ddd�Z dd� Z ee�ddd��ZdS )� XportReaderN� ISO-8859-1c C sn || _ d| _|| _|| _t|d|dd�| _ttt | jj �| _ y | � � W n t k rh | � � � Y nX d S )Nr �rbF)�encodingZis_text)� _encoding� _lines_read�_index� _chunksizer �handlesr r �bytes�handle�filepath_or_buffer� _read_header� Exception�close)�selfrC �indexr; � chunksizer r r �__init__� s zXportReader.__init__c C s | j �� d S )N)r@ rF )rG r r r rF s zXportReader.closec C s | j �d��� S )N�P )rC �read�decode)rG r r r �_get_row s zXportReader._get_rowc C s: | j �d� | �� }|tkr$td��| �� }ddgddgddgddgd d gg}t||�}|d d krltd ��t|d �|d < || _| �� }t|d d � �|d < | �� }| �� }|�t �}|t k} |r�| s�td��t |dd� �} ddgddgddgddgddgddgd d gg} t| �� | �} d d gdd gddgddgg} | � t| �� | �� t| d �| d < t| d �| d < | | _ ddd�} t | �� dd� �}| | }|d �r�|d|d 7 }| j �|�}g }d}x�t|�| k�r�|d | � || d � }}|�d �}t�d |�}ttt|��}|d= | |d |d < |d }|d dk�rh|d k �sT|dk�rhd!|� d"�}t|��x>|�� D ]2\}}y|�� ||< W n tk �r� Y nX �qrW ||d 7 }||g7 }�q�W | �� }|tk�s�td#��|| _|| _ | j � � | _ | � � | _ d$d%� | jD �| _!d&d%� t"| j�D �}t#�$|�} | | _%d S )'Nr z#Header record is not an XPORT file.�prefixr0 �versionr ZOSr �created� zSAS SAS SASLIBz!Header record has invalid prefix.�modifiedzMember header not found�������set_nameZsasdatar �( �type�numeric�char)r+ r, �6 �: rK � z>hhhh8s40s8shhh2s8shhl52sr r r, zFloating field width z is not between 2 and 8.z Observation header not found.c S s g | ]}|d � � �qS )r )rM )�.0�xr r r � <listcomp>r s z,XportReader._read_header.<locals>.<listcomp>c S s, g | ]$\}}d t |� dt |d � f�qS )r r r )�str)r^ �i�fieldr r r r` v s )&rC �seekrN �_correct_line1r r r � file_info� startswith�_correct_header1�_correct_header2�int�update� member_inforL r$ �ljust�struct�unpack�dict�zip� _fieldkeys� TypeError�itemsr �AttributeError�_correct_obs_header�fields� record_length�tell� record_start� _record_count�nobs�columns� enumerater" r �_dtype) rG �line1�line2�fifrf �line3�header1�header2Z headflag1Z headflag2ZfieldnamelengthZmemrl �typesZ fieldcountZ datalengthZ fielddatarw Z obs_lengthZ fieldbytesZ fieldstructrc �fl�msg�k�v�headerZdtypelr r r r rD s� " " zXportReader._read_headerc C s | j | jp dd�S )Nr+ )�nrows)rL r? )rG r r r �__next__| s zXportReader.__next__)r c C s� | j �dd� | j �� | j }|d dkr4t�d� | jdkrV| j �| j� || j S | j �dd� | j �d�}tj |tj d�}t� |dk�}t |�dkr�d}n dt |� }| j �| j� || | j S ) z� Get number of records in file. This is maybe suboptimal because we have to seek to the end of the file. Side effect: returns file position to record_start. r r, rK zxport file may be corruptedi����)r l @@� r ) rC rd ry rz �warnings�warnrx rL r" � frombufferZuint64Z flatnonzeror$ )rG Ztotal_records_lengthZlast_card_bytesZ last_cardZixZtail_padr r r r{ s zXportReader._record_countc C s |dkr| j }| j|d�S )a Reads lines from Xport file and returns as dataframe Parameters ---------- size : int, defaults to None Number of lines to read. If None, reads whole file. Returns ------- DataFrame N)r� )r? rL )rG �sizer r r � get_chunk� s zXportReader.get_chunkc C sl |j dd�}|d dk|d dk@ |d dk@ }|d dk|d d k@ |d d kB |d d kB }||M }|S ) Nz u1,u1,u2,u4)r r* r �f2Zf3r! r2 �Z �_ �. )r% )rG r& r� �missZmiss1r r r �_missing_double� s $0zXportReader._missing_doublec sj |d kr� j }t|� j � j �}|� j }|dkr>� �� t�� j�|�}tj |� j |d�}t j t |�d�}x�t� j�D ]�\}}|dt|� } � j| d } | dkr�t| � j| d �} � �| �} t| �} tj| | < n@� j| d dk�rd d � | D �} � jd k �r� fd d �| D �} | ||< qxW � jd k�rLt �t � j� j| ��|_n |�� j�}� j|7 _|S ) Nr )r �count)rH r r rY r rZ c S s g | ] }|� � �qS r )�rstrip)r^ �yr r r r` � s z$XportReader.read.<locals>.<listcomp>c s g | ]}|� � j��qS r )rM r< )r^ r� )rG r r r` � s )r| �minr= rx rF � StopIterationrC rL r" r� r r Z DataFrame�ranger~ r} ra rw r) r� r7 �nanr< r> �IndexrH Z set_index) rG r� Z read_lines�read_len�raw�dataZdf�jr_ r& r r� r� r )rG r rL � s8 zXportReader.read)Nr9 N)N)N)�__name__� __module__� __qualname__�_xport_reader_doc�__doc__rJ rF rN rD r� rj r{ r� r� r �_read_method_docrL r r r r r8 � s h# r8 )&r� � collectionsr r rn �typingr r r� �numpyr" Zpandas.util._decoratorsr Zpandasr Zpandas.io.commonr Zpandas.io.sas.sasreaderr re rh ri rv rr Z_base_params_docZ _params2_docZ_format_params_docZ _iterator_docZ _read_sas_docr� r� ra r r r) r7 �Iteratorr8 r r r r �<module> sT 9
| ver. 1.4 |
Github
|
.
| PHP 8.3.23 | Генерация страницы: 0.03 |
proxy
|
phpinfo
|
Настройка