Файловый менеджер - Редактировать - /home/digitalm/venv/lib/python3.7/site-packages/pandas/tests/io/__pycache__/test_parquet.cpython-37.pyc
Назад
B �5�g� � @ s� d Z ddlZddlmZ ddlZddlZddlmZmZ ddl Z ddl Z ddl m Z ddlmZmZ ddlmZmZ ddlm mZ ddlZddlmZ ddl m Z dd l m Z m!Z!m"Z"m#Z#m$Z$ yddl%Z%d Z&W n e'k r� d Z&Y nX y0e� � ed d e(d� ddl)Z)W dQ R X d Z*W n e'k �r6 d Z*Y nX e j+�d�Z,e j-e j.de j+j/e* �pfe d�dkdd�d�e j.de j+j/e& dd�d�gd�dd� �Z0e j-dd � �Z1e j-d d � �Z2e j-d d � �Z3e j-d!d"� �Z4e j-d#d$� �Z5e j-ej�6ej7j8�ej�6ej7j9�ej�6ej7j:�ej�;d%d&�ej�;d'd&�ej�;d(d&�ej�;d)d&�gd�d*d+� �Z<dHd-d.�Z=d/d0� Z>d1d2� Z?d3d4� Z@d5d6� ZAd7d8� ZBd9d:� ZCd;d<� ZDd=d>� ZEG d?d@� d@�ZFG dAdB� dBeF�ZGe j+�dC�G dDdE� dEeF��ZHG dFdG� dGeF�ZIdS )Iz test parquet compat � N)�BytesIO)�catch_warnings�filterwarnings)� get_option)�PY38�is_platform_windows)�pa_version_under1p0�pa_version_under2p0)�Version)�FastParquetImpl� PyArrowImpl� get_engine� read_parquet� to_parquetTF�ignorez `np.bool`)�categoryz4ignore:RangeIndex.* is deprecated:DeprecationWarning� fastparquetzmode.data_manager�arrayz4fastparquet is not installed or ArrayManager is used)�reason)Zmarks�pyarrowzpyarrow is not installed)�paramsc C s | j S )N)�param)�request� r �R/home/digitalm-up/venv/lib/python3.7/site-packages/pandas/tests/io/test_parquet.py�engine@ s r c C s t st�d� dS )Nzpyarrow is not installedr )� _HAVE_PYARROW�pytest�skipr r r r �paU s r c C s* t st�d� ntd�dkr&t�d� dS )Nz fastparquet is not installedzmode.data_managerr z.ArrayManager is not supported with fastparquetr )�_HAVE_FASTPARQUETr r r r r r r �fp\ s r! c C s t �dddgdd��S )N� � � �foo)�A�B)�pd� DataFramer r r r � df_compate s r* c C sD t �td�ttdd��tjdddd�dd dgt jd d d �d ��} | S )N�abcr" � g @g @�float64)�dtypeTF�20130101r$ )�periods)�a�b�d�e�f)r( r) �list�range�np�arange� date_range)�dfr r r �df_cross_compatj s r< c C s� t �td�dtjdgdd dgdddgddd gttd d ��t�d d ��d�tjdddd�dtjdgdddgt jdd d�t � d�t j t � d�gd� �S )Nr+ r1 �cs foos bars bazr% �bar�bazr" r, r$ � �u1g @g @r- )r. g @g @TFr/ )r0 Z20130103) �stringZstring_with_nanZstring_with_none�bytes�unicode�intZuint�floatZfloat_with_nan�bool�datetimeZdatetime_with_nat) r( r) r6 r8 �nanr7 r9 �astyper: � TimestampZNaTr r r r �df_full| s rL z2019-01-04T16:41:24+0200z%Y-%m-%dT%H:%M:%S%zz2019-01-04T16:41:24+0215z2019-01-04T16:41:24-0200z2019-01-04T16:41:24-0215c C s | j S )N)r )r r r r �timezone_aware_date_list� s rM r# c s� �p ddi��pi ��dkr ��|r4|�d< |�d< � �������fdd�} �dkrtt �� ��| | � W dQ R X n| | � dS )a Verify parquet serializer and deserializer produce the same results. Performs a pandas to disk and disk to pandas round trip, then compares the 2 resulting DataFrames to verify equality. Parameters ---------- df: Dataframe engine: str, optional 'pyarrow' or 'fastparquet' path: str, optional write_kwargs: dict of str:str, optional read_kwargs: dict of str:str, optional expected: DataFrame, optional Expected deserialization result, otherwise will be equal to `df` check_names: list of str, optional Closed set of column names to be compared check_like: bool, optional If True, ignore the order of index & columns. repeat: int, optional How many times to repeat the test � compressionNr c sZ xTt | �D ]H}�j�f�� tdd�� t�f��}W d Q R X tj�|��� d� q W d S )NT)�record)� check_names� check_like� check_dtype)r7 r r r �tm�assert_frame_equal)�repeat�_�actual)rR rQ rP r; �expected�path� read_kwargs� write_kwargsr r �compare� s z!check_round_trip.<locals>.compare)rS � ensure_clean) r; r rY r[ rZ rX rP rQ rR rU r\ r )rR rQ rP r; rX rY rZ r[ r �check_round_trip� s " r^ c C s. d}t jt|d�� t| dd� W d Q R X d S )Nz.engine must be one of 'pyarrow', 'fastparquet')�matchr% r> )r �raises� ValueErrorr^ )r* �msgr r r �test_invalid_engine� s rc c C s$ t �dd�� t| � W d Q R X d S )Nzio.parquet.enginer )r( �option_contextr^ )r* r r r r �test_options_py� s re c C s$ t �dd�� t| � W d Q R X d S )Nzio.parquet.enginer )r( rd r^ )r* r! r r r �test_options_fp� s rf c C s$ t �dd�� t| � W d Q R X d S )Nzio.parquet.engine�auto)r( rd r^ )r* r! r r r r �test_options_auto� s rh c C s t td�t�st�t td�t�s$t�t�dd��< t td�t�sDt�t td�t�sVt�t td�t�sht�W d Q R X t�dd��< t td�t�s�t�t td�t�s�t�t td�t�s�t�W d Q R X t�dd��> t td�t�s�t�t td�t�s�t�t td�t��st�W d Q R X d S )Nr r zio.parquet.enginerg )� isinstancer r �AssertionErrorr r( rd )r! r r r r �test_options_get_engine� s rk c C s. ddl m} | �d�}| �d�}ts(dnttj�t|�k }tsBdnttj�t|�k }to\| }tof| }|�s*|�s*|r�d|� d�}t j t |d�� t d � W d Q R X n&d }t j t |d�� t d � W d Q R X |�rd|� d �}t j t |d�� t d � W d Q R X n&d }t j t |d�� t d � W d Q R X d S ) Nr )�VERSIONSr r FzPandas requires version .z. or newer of .pyarrow.)r_ rg z%Missing optional dependency .pyarrow.z. or newer of .fastparquet.z)Missing optional dependency .fastparquet.) Zpandas.compat._optionalrl �getr r r � __version__r r r r` � ImportErrorr )rl Z pa_min_verZ fp_min_verZhave_pa_bad_versionZhave_fp_bad_versionZhave_usable_paZhave_usable_fpr_ r r r �"test_get_engine_auto_error_message s0 rp c C sj | }t �� �T}|j||d d� t||d�}t �||� t||ddgd�}t �||ddg � W d Q R X d S )N)r rN )r r1 r3 )r �columns)rS r] r r rT )r< r r! r; rY �resultr r r �test_cross_engine_pa_fp> s rs c C s� |}t �� �j}|j||d d� tdd��D t||d�}t �||� t||ddgd�}t �||ddg � W d Q R X W d Q R X d S )N)r rN T)rO )r r1 r3 )r rq )rS r] r r r rT )r r< r r! r; rY rr r r r �test_cross_engine_fp_paL s rt c @ s* e Zd Zdd� Zdd� Zejdd� �ZdS )�Basec C sB t �� �0}tj||d�� t|||d d� W d Q R X W d Q R X d S )N)r_ )rN )rS r] r r` r )�selfr; r �exc�err_msgrY r r r �check_error_on_write[ s zBase.check_error_on_writec C s> t �� �,}t �|�� t|||d d� W d Q R X W d Q R X d S )N)rN )rS r] Zexternal_error_raisedr )rv r; r rw rY r r r � check_external_error_on_writea s z"Base.check_external_error_on_writec C s. |dkrt �|� d}t|�}t�||� d S )Nrg zfhttps://raw.githubusercontent.com/pandas-dev/pandas/master/pandas/tests/io/data/parquet/simple.parquet)r � importorskipr rS rT )rv r* r �urlr; r r r �test_parquet_read_from_urlg s z Base.test_parquet_read_from_urlN)�__name__� __module__� __qualname__ry rz rS �networkr} r r r r ru Z s ru c @ s� e Zd Zdd� Zdd� Zdd� Zej�ddd d d g�d d � �Z dd� Z dd� Z dd� Z dd� Z dd� Zdd� Zdd� Zd d � Zd d � Zd d!� Zd"d#� ZdS )$� TestBasicc C sN xHt �dddg�ddt �d�t�dddg�gD ]}d}| �||t|� q.W d S )Nr" r# r$ r% r/ z+to_parquet only supports IO with DataFrames)r( �SeriesrK r8 r ry ra )rv r �objrb r r r � test_errort s zTestBasic.test_errorc C s6 t �td�ttdd��d��}ddg|_t||� d S )Nr+ r" r, )rB rE r% r> )r( r) r6 r7 rq r^ )rv r r; r r r �test_columns_dtypes s z TestBasic.test_columns_dtypesc C s� t �td�ttdd��d��}d}ddg|_| �||t|� ddg|_| �||t|� t�d dddd�t�d dddd�g|_| �||t|� d S ) Nr+ r" r, )rB rE z%parquet must have string column namesr s foos bari� )r( r) r6 r7 rq ry ra rH )rv r r; rb r r r �test_columns_dtypes_invalid� s z%TestBasic.test_columns_dtypes_invalidrN N�gzip�snappy�brotlic C sP |dkrt �d� n|dkr&t �d� t�ddddgi�}t||d|id� d S ) Nr� r� r& r" r# r$ rN )r[ )r r{ r( r) r^ )rv r rN r; r r r �test_compression� s zTestBasic.test_compressionc C sJ t �td�ttdd��d��}t �dtd�i�}t|||ddgid� d S )Nr+ r" r, )rB rE rB rq )rX rZ )r( r) r6 r7 r^ )rv r r; rX r r r �test_read_columns� s zTestBasic.test_read_columnsc C s� |dk}t �ddddgi�}t||� dddgt jddd�td �dddgg}x:|D ]2}||_t|t j�rv|j�d �|_t|||d � qRW d ddg|_d |j_ t||� d S ) Nr r& r" r# r$ r, r/ )r0 r+ )rP r r% ) r( r) r^ r: r6 �indexri Z DatetimeIndex� _with_freq�name)rv r rP r; Zindexesr� r r r �test_write_index� s zTestBasic.test_write_indexc C s> |}t �ddddgi�}t j�dddg�}||_t||� d S )Nr& r" r# r$ )r1 r" )r1 r# )r2 r" )r( r) � MultiIndex� from_tuplesr� r^ )rv r r r; r� r r r �test_write_multiindex� s z TestBasic.test_write_multiindexc C s� |}t jdddd�}t jtj�dt|� d�td�d�}t jj d d g|gd d gd �}|j d d �}x@||gD ]4}||_ t ||� t ||dddgi|ddg d� qjW d S )Nz 01-Jan-2018z 01-Dec-2018ZMS)�freqr# r$ �ABC)rq �Level1�Level2�level�date)�namesrq r& r' )rZ rX ) r( r: r) r8 �random�randn�lenr6 r� Z from_product�copyr� r^ )rv r r �datesr; Zindex1�index2r� r r r � test_multiindex_with_columns� s $ z&TestBasic.test_multiindex_with_columnsc C s� t �dddgdddgd��}d dd �}|jd d �}t||||d � t jdddgdddgd�d ddgd�}t||||d � ddddddddgddddddddgg}t jttd��dd� td�D �d�|d�}|jd d �}t||||d � d S )Nr" r# r$ �q�r�s)r1 r2 F)rN r� T)�drop)r[ rX ZzyxZwvuZtsr)r� r> r? r% �qux�one�two� c S s g | ] }| �qS r r )�.0�ir r r � <listcomp>� s z7TestBasic.test_write_ignoring_index.<locals>.<listcomp>)r� r� )r( r) Z reset_indexr^ r6 r7 )rv r r; r[ rX �arraysr r r �test_write_ignoring_index� s "& z#TestBasic.test_write_ignoring_indexc C sB t j�dddg�}t jtj�dd�|d�}d}| �||t|� d S )N)r1 r" )r1 r# )r2 r" r, r$ )rq zY\s*parquet must have string column names for all values in\s*each level of the MultiIndex) r( r� r� r) r8 r� r� ry ra )rv r Z mi_columnsr; rb r r r � test_write_column_multiindex� s z&TestBasic.test_write_column_multiindexc C sh |}ddddddddgddddddddgg}t jtj�dd�|d�}d d g|j_d }| �||t|� d S ) Nr> r? r% r� r" r# r� )rq r� r� zY\s*parquet must have string column names for all values in\s*each level of the MultiIndex) r( r) r8 r� r� rq r� ry ra )rv r r r� r; rb r r r �&test_write_column_multiindex_nonstring s z0TestBasic.test_write_column_multiindex_nonstringc C s^ |}ddddddddgddddddddgg}t jtj�dd�|d�}d d g|j_t||� d S ) Nr> r? r% r� r� r� r� )rq Z ColLevel1Z ColLevel2)r( r) r8 r� r� rq r� r^ )rv r r r� r; r r r �#test_write_column_multiindex_string s z-TestBasic.test_write_column_multiindex_stringc C s> |}ddddg}t jtj�dd�|d�}d|j_t||� d S ) Nr> r? r% r� r� r, )rq Z StringCol)r( r) r8 r� r� rq r� r^ )rv r r r� r; r r r � test_write_column_index_string* s z(TestBasic.test_write_column_index_stringc C sH |}ddddg}t jtj�dd�|d�}d|j_d}| �||t|� d S ) Nr" r# r$ r, r� )rq Z NonStringColz%parquet must have string column names) r( r) r8 r� r� rq r� ry ra )rv r r r� r; rb r r r �!test_write_column_index_nonstring6 s z+TestBasic.test_write_column_index_nonstringc C sx dd l m} |dkr t�d� t�t�dddd gd�t�dddd gd�t�d d d d g�t�d d d d g�t�ddddgd�d��}t�� �,}|� ||� t ||d�}t ||d d�}W d Q R X |d j t � d�ks�t �t�tjdddd gdd�tjdddd gdd�tjd d d d gdd�tjd d d d gdd�tjddddgdd�d��}|dk�rh|jd dd�}|jd dd�}t�||� d S )Nr r z.Fastparquet nullable dtype support is disabledr" r# r$ Zint64Zuint8r1 r2 r= TFr, )r1 r2 r= r3 r4 )r )r �use_nullable_dtypesr- �Int64)r. ZUInt8rB �boolean)Zaxis)�pyarrow.parquet�parquetr r r �tabler rS r] Z write_tabler r. r8 rj r( r) r� rT )rv r �pqr� rY Zresult1Zresult2rX r r r �test_use_nullable_dtypesB s0 z"TestBasic.test_use_nullable_dtypes)r~ r r� r� r� r� r �mark� parametrizer� r� r� r� r� r� r� r� r� r� r� r� r r r r r� s s r� z8ignore:CategoricalBlock is deprecated:DeprecationWarningc @ sR e Zd Zdd� Zdd� Zdd� Zdd� Zd d � Zd d � Ze j j e � oFe d dd�dd� �Zdd� Ze�d�e j �ddgg g�dd� ��Ze�d�dd� �Ze�d�d d � �Zd d � Zd d!� Ze j �d"eejg�d#d$� �Zd%d&� Z d'd(� Z e�d�d)d*� �Z ejdd+d,�d-d.� �Z e�d�d/d0� �Z d1d2� Z!d3d4� Z"ejdd+d,�d5d6� �Z#d7d8� Z$d9S ):�TestParquetPyArrowc C sB |}t jdddd�}|�d �}||d< dd dg|d< t||� d S )Nr/ r$ zEurope/Brussels)r0 �tz� datetime_tzTZbool_with_none)r( r: r� r^ )rv r rL r; �dtir r r � test_basicn s z TestParquetPyArrow.test_basicc C s< |}t jdddd�|d< t|||ddg dddgid � d S ) Nr/ r$ zEurope/Brussels)r0 r� r� rB rE rq )rX rZ )r( r: r^ )rv r rL r; r r r �test_basic_subset_columnsz s z,TestParquetPyArrow.test_basic_subset_columnsc C s: |j |d�}t|t�st�t|�}t|�}t�||� d S )N)r )r ri rC rj r r rS rT )rv r rL Z buf_bytesZ buf_stream�resr r r �*test_to_bytes_without_path_or_buf_provided� s z=TestParquetPyArrow.test_to_bytes_without_path_or_buf_providedc C s8 t jt�d��dd�td�d��� }| �||td� d S )N� r, r$ �aaa)rq z Duplicate column names found) r( r) r8 r9 �reshaper6 r� ry ra )rv r r; r r r �test_duplicate_columns� s $z)TestParquetPyArrow.test_duplicate_columnsc C sN t �dt jddd�i�}| �||t� t �ddddgi�}| �||tj� d S )Nr1 z1 dayr$ )r0 r" g @)r( r) �timedelta_rangerz �NotImplementedErrorr ZArrowException)rv r r; r r r �test_unsupported� s z#TestParquetPyArrow.test_unsupportedc C sx t �� }t �td��|d< t jddddd dgt �dddg�d�|d< t jddddddgddd gd d �|d< t||� d S ) NZabcdefr1 r> r% r? )r. r2 r= r3 T)� categoriesZordered)r( r) � Categoricalr6 ZCategoricalDtyper^ )rv r r; r r r �test_categorical� s "z#TestParquetPyArrow.test_categoricalz localhost connection rejectedF)r �strictc C s4 t �d�}|jf |�}d|i}t||d||d� d S )N�s3fs� filesystemzpandas-test/pyarrow.parquet)rY rZ r[ )r r{ Z S3FileSystemr^ )rv r* � s3_resourcer �s3sor� Zs3�kwr r r � test_s3_roundtrip_explicit_fs� s z0TestParquetPyArrow.test_s3_roundtrip_explicit_fsc C s d|i}t ||d||d� d S )N�storage_optionsz s3://pandas-test/pyarrow.parquet)rY rZ r[ )r^ )rv r* r� r r� r r r �test_s3_roundtrip� s z$TestParquetPyArrow.test_s3_roundtripr� � partition_colr& c C sV |� � }|r.tstrdnd}|| �|�||< t|||dd|i|d |d�ddd� d S ) NZint32r z s3://pandas-test/parquet_dirr� )�partition_colsrN r� Tr" )rX rY rZ r[ rQ rU )r� r r rJ r^ )rv r* r� r r� r� Z expected_dfZpartition_col_typer r r �test_s3_roundtrip_for_dir� s" z,TestParquetPyArrow.test_s3_roundtrip_for_dirr c C s( t � }|�|� t|�}t�||� d S )N)r r r rS rT )rv r* �bufferZ df_from_bufr r r � test_read_file_like_obj_support s z2TestParquetPyArrow.test_read_file_like_obj_supportc C sb |� dd� |� dd� tjtdd�� td� W d Q R X tjtdd�� |�d� W d Q R X d S )N�HOMEZ TestingUser� USERPROFILEz.*TestingUser.*)r_ z~/file.parquet)Zsetenvr r` �OSErrorr r )rv r* � monkeypatchr r r �test_expand_user s z#TestParquetPyArrow.test_expand_userc C s� ddg}|}t �� �l}|j||d d� dd lm} |j|dd�}t|jj�dksTt �|jjt |�ksht �t |�j |j ks|t �W d Q R X d S )NrG rE )r� rN r F)�validate_schemar# ) rS �ensure_clean_dirr r� r� �ParquetDatasetr� � partitions�partition_namesrj �setr �shape)rv r rL r� r; rY r� �datasetr r r � test_partition_cols_supported s z0TestParquetPyArrow.test_partition_cols_supportedc C s� d}|g}|}t �� �l}|j||d d� dd lm} |j|dd�}t|jj�dksVt �|jjt |�ksjt �t |�j |j ks~t �W d Q R X d S )NrG )r� rN r F)r� r" ) rS r� r r� r� r� r� r� r� rj r� r r� ) rv r rL r� �partition_cols_listr; rY r� r� r r r �test_partition_cols_string s z-TestParquetPyArrow.test_partition_cols_string� path_typec C sP d}|g}|}t �� �0}||�}|j||d� t|�j|jksBt�W d Q R X d S )Nr' )r� )rS r� r r r� rj ) rv r r* r� r� r� r; Zpath_strrY r r r �test_partition_cols_pathlib+ s z.TestParquetPyArrow.test_partition_cols_pathlibc C s t �� }t||� d S )N)r( r) r^ )rv r r; r r r �test_empty_dataframe8 s z'TestParquetPyArrow.test_empty_dataframec C sV dd l }t�dddgi�}|�|jd|�� d�g�}|�t�}t||d|i|d� d S )Nr �xr" )�type�schema)r[ rX ) r r( r) r� �fieldZbool_rJ rG r^ )rv r r r; r� Zout_dfr r r �test_write_with_schema= s z)TestParquetPyArrow.test_write_with_schemac C sz t �t jdddgdd�t jdddgdd�t jdd dgd d�d ��}t||� t �dt jdddd gdd�i�}t||� d S ) Nr" r# r$ r� )r. ZUInt32r1 r= rB )r1 r2 r= )r( r) r� r^ )rv r r; r r r � test_additional_extension_arraysE s z3TestParquetPyArrow.test_additional_extension_arraysz1.0.0)� min_versionc C sV t �dt jdd dgdd�i�}t �d|��" t|||�d|� d��d� W d Q R X d S ) Nr1 r= zstring[pyarrow])r. �string_storagezstring[�])rX )r( r) r� rd r^ rJ )rv r r� r; r r r � test_pyarrow_backed_string_arrayU s z3TestParquetPyArrow.test_pyarrow_backed_string_arrayc C s( t �dt jdddd�i�}t||� d S )Nr3 z 2012-01-01r$ �D)r0 r� )r( r) � period_ranger^ )rv r r; r r r � test_additional_extension_types\ s z2TestParquetPyArrow.test_additional_extension_typesc C s0 t �dt jdddd�i�}t||ddid� d S ) Nr1 z 2017-01-01Z1n� )r� r0 �versionz2.0)r[ )r( r) r: r^ )rv r r; r r r �test_timestamp_nanosecondsi s z-TestParquetPyArrow.test_timestamp_nanosecondsc C s: t s t�� d|g }tj|d|id�}t||dd� d S )N� � index_as_col)r� �dataF)rR )r r r r( r) r^ )rv r rM �idxr; r r r �test_timezone_aware_indexo s z,TestParquetPyArrow.test_timezone_aware_indexc C s^ t �dttdd��i�}t�� �$}|�||� t||dgdd�}W d Q R X t|�dksZt �d S )Nr1 r r$ )r1 z==r F)�filtersZuse_legacy_datasetr" ) r( r) r6 r7 rS r] r r r� rj )rv r r; rY rr r r r �test_filter_row_groups� s z)TestParquetPyArrow.test_filter_row_groupsc C s~ t jtj�dd�dddgd�}t�� � }|�||� t||�}W d Q R X |rdt |j t j j j �szt�nt |j t j j j�szt�d S )Nr� r$ r& r' �C)rq )r( r) r8 r� r� rS r] r r ri Z_mgr�coreZ internalsZ ArrayManagerrj Z BlockManager)rv r Zusing_array_managerr; rY rr r r r �test_read_parquet_manager� s z,TestParquetPyArrow.test_read_parquet_managerN)%r~ r r� r� r� r� r� r� r� r r� Zxfailr r r� r� �tdZ skip_if_nor� r� r� r� r� r� �str�pathlib�Pathr� r� r� r� r� r� r r r r r r r r r� l s< ) r� c @ s� e Zd Zdd� Zejjdd�dd� �Zdd� Zd d � Z d d � Z d d� Z dd� Z dd� Z dd� Zdd� Zdd� Zdd� Zdd � Zd d � Zd S ) �TestParquetFastParquetc C sF |}t jdddd�}|�d �}||d< t jddd�|d< t||� d S ) Nr/ r$ z US/Eastern)r0 r� r� z1 day)r0 � timedelta)r( r: r� r� r^ )rv r! rL r; r� r r r r� � s z!TestParquetFastParquet.test_basicz not supported)r c C s< t jt�d��dd�td�d��� }d}| �||t|� d S )Nr� r, r$ r� )rq z9Cannot create parquet dataset with duplicate column names) r( r) r8 r9 r� r6 r� ry ra )rv r! r; rb r r r r� � s $z-TestParquetFastParquet.test_duplicate_columnsc C sB t �ddd dgi�}t jddtjdgidd�}t|||dd� d S ) Nr1 TFg �?g Zfloat16)r. )rX rR )r( r) r8 rI r^ )rv r! r; rX r r r �test_bool_with_none� s z*TestParquetFastParquet.test_bool_with_nonec C sV t �dt jdddd�i�}| �||td � t �ddddgi�}d}| �||t|� d S ) Nr1 Z2013�Mr$ )r� r0 r" g @z"Can't infer object conversion type)r( r) r� ry ra )rv r! r; rb r r r r� � s z'TestParquetFastParquet.test_unsupportedc C s& t �dt �td��i�}t||� d S )Nr1 r+ )r( r) r� r6 r^ )rv r! r; r r r r� � s z'TestParquetFastParquet.test_categoricalc C sf dt tdd��i}t�|�}t�� �(}|j||d dd� t||dgd�}W d Q R X t|�dksbt �d S )Nr1 r r$ r" )rN Zrow_group_offsets)r1 z==r )r ) r6 r7 r( r) rS r] r r r� rj )rv r! r3 r; rY rr r r r r � s z-TestParquetFastParquet.test_filter_row_groupsc C s t ||dd|id |d�d� d S )Nz$s3://pandas-test/fastparquet.parquetr� )rN r� )rY rZ r[ )r^ )rv r* r� r! r� r r r r� � s z(TestParquetFastParquet.test_s3_roundtripc C sl ddg}|}t �� �N}|j|d|d d� tj�|�s8t�dd l}|�|d�j }t |�dks^t�W d Q R X d S )NrG rE r )r r� rN r Fr# ) rS r� r �osrY �existsrj r � ParquetFile�catsr� )rv r! rL r� r; rY r �actual_partition_colsr r r r� � s z4TestParquetFastParquet.test_partition_cols_supportedc C sh d}|}t �� �N}|j|d|d d� tj�|�s4t�dd l}|�|d�j }t |�dksZt�W d Q R X d S )NrG r )r r� rN r Fr" ) rS r� r r rY r rj r r r r� )rv r! rL r� r; rY r r r r r r� � s z1TestParquetFastParquet.test_partition_cols_stringc C sl ddg}|}t �� �N}|j|dd |d� tj�|�s8t�dd l}|�|d�j }t |�dks^t�W d Q R X d S )NrG rE r )r rN � partition_onr Fr# ) rS r� r r rY r rj r r r r� )rv r! rL r� r; rY r r r r r �test_partition_on_supported� s z2TestParquetFastParquet.test_partition_on_supportedc C sV ddg}|}d}t jt|d��. t�� �}|j|dd ||d� W d Q R X W d Q R X d S )NrG rE zYCannot use both partition_on and partition_cols. Use partition_cols for partitioning data)r_ r )r rN r r� )r r` ra rS r� r )rv r! rL r� r; rb rY r r r �3test_error_on_using_partition_cols_and_partition_on s zJTestParquetFastParquet.test_error_on_using_partition_cols_and_partition_onc C s* t �� }|�� }d|j_t|||d� d S )Nr� )rX )r( r) r� r� r� r^ )rv r! r; rX r r r r� s z+TestParquetFastParquet.test_empty_dataframec C s> d|g }t j|d|id�}|�� }d|j_t|||d� d S )Nr r )r� r r� )rX )r( r) r� r� r� r^ )rv r! rM r r; rX r r r r ! s z0TestParquetFastParquet.test_timezone_aware_indexc C s\ t �dddgi�}t�� �8}|�|� tjtdd�� t|ddd� W d Q R X W d Q R X d S ) Nr1 r" r# z!not supported for the fastparquet)r_ r T)r r� ) r( r) rS r] r r r` ra r )rv r� r! r; rY r r r �&test_use_nullable_dtypes_not_supported* s z=TestParquetFastParquet.test_use_nullable_dtypes_not_supportedN)r~ r r� r� r r� r r� r r� r� r r� r� r� r r r� r r r r r r r � s r ) NNNNNTFTr# )J�__doc__rH �ior r r �warningsr r �numpyr8 r Zpandas._configr Z pandas.compatr r Zpandas.compat.pyarrowr r Z pandas.util._test_decorators�utilZ_test_decoratorsr Zpandasr( Zpandas._testingZ_testingrS Zpandas.util.versionr Zpandas.io.parquetr r r r r r r ro �DeprecationWarningr r r� Z pytestmarkZfixturer Zskipifr r r! r* r< rL �now�timezone�utc�min�max�strptimerM r^ rc re rf rh rk rp rs rt ru r� r� r r r r r �<module> s� 7+ z /
| ver. 1.4 |
Github
|
.
| PHP 8.3.23 | Генерация страницы: 0.03 |
proxy
|
phpinfo
|
Настройка